-- Logs begin at Sat 2022-08-06 21:13:00 UTC, end at Sat 2022-08-06 22:36:16 UTC. -- Aug 06 21:49:10.611222 dv-cinder-849846-7-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 06 21:49:10.620436 dv-cinder-849846-7-iscsi devstack@keystone.service[80412]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 06 21:49:10.622466 dv-cinder-849846-7-iscsi devstack@keystone.service[80412]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sat Aug 6 21:49:10 2022] *** Aug 06 21:49:10.622466 dv-cinder-849846-7-iscsi devstack@keystone.service[80412]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 06 21:49:10.622466 dv-cinder-849846-7-iscsi devstack@keystone.service[80412]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 06 21:49:10.622466 dv-cinder-849846-7-iscsi devstack@keystone.service[80412]: nodename: dv-cinder-849846-7-iscsi Aug 06 21:49:10.622466 dv-cinder-849846-7-iscsi devstack@keystone.service[80412]: machine: x86_64 Aug 06 21:49:10.622466 dv-cinder-849846-7-iscsi devstack@keystone.service[80412]: clock source: unix Aug 06 21:49:10.622466 dv-cinder-849846-7-iscsi devstack@keystone.service[80412]: pcre jit disabled Aug 06 21:49:10.622466 dv-cinder-849846-7-iscsi devstack@keystone.service[80412]: detected number of CPU cores: 4 Aug 06 21:49:10.622466 dv-cinder-849846-7-iscsi devstack@keystone.service[80412]: current working directory: / Aug 06 21:49:10.622466 dv-cinder-849846-7-iscsi devstack@keystone.service[80412]: detected binary path: /usr/bin/uwsgi-core Aug 06 21:49:10.623012 dv-cinder-849846-7-iscsi devstack@keystone.service[80412]: your processes number limit is 47680 Aug 06 21:49:10.623012 dv-cinder-849846-7-iscsi devstack@keystone.service[80412]: your memory page size is 4096 bytes Aug 06 21:49:10.623012 dv-cinder-849846-7-iscsi devstack@keystone.service[80412]: detected max file descriptor number: 2048 Aug 06 21:49:10.623012 dv-cinder-849846-7-iscsi devstack@keystone.service[80412]: lock engine: pthread robust mutexes Aug 06 21:49:10.623012 dv-cinder-849846-7-iscsi devstack@keystone.service[80412]: thunder lock: enabled Aug 06 21:49:10.623012 dv-cinder-849846-7-iscsi devstack@keystone.service[80412]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 06 21:49:10.623012 dv-cinder-849846-7-iscsi devstack@keystone.service[80412]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 06 21:49:10.662088 dv-cinder-849846-7-iscsi devstack@keystone.service[80412]: Python main interpreter initialized at 0x55cbdb8299c0 Aug 06 21:49:10.662088 dv-cinder-849846-7-iscsi devstack@keystone.service[80412]: python threads support enabled Aug 06 21:49:10.662088 dv-cinder-849846-7-iscsi devstack@keystone.service[80412]: your server socket listen backlog is limited to 100 connections Aug 06 21:49:10.662088 dv-cinder-849846-7-iscsi devstack@keystone.service[80412]: your mercy for graceful operations on workers is 90 seconds Aug 06 21:49:10.662433 dv-cinder-849846-7-iscsi devstack@keystone.service[80412]: mapped 403077 bytes (393 KB) for 2 cores Aug 06 21:49:10.662513 dv-cinder-849846-7-iscsi devstack@keystone.service[80412]: *** Operational MODE: preforking *** Aug 06 21:49:10.662776 dv-cinder-849846-7-iscsi devstack@keystone.service[80412]: *** uWSGI is running in multiple interpreter mode *** Aug 06 21:49:10.662776 dv-cinder-849846-7-iscsi devstack@keystone.service[80412]: spawned uWSGI master process (pid: 80412) Aug 06 21:49:10.662982 dv-cinder-849846-7-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 06 21:49:10.668439 dv-cinder-849846-7-iscsi devstack@keystone.service[80412]: spawned uWSGI worker 1 (pid: 80423, cores: 1) Aug 06 21:49:10.668439 dv-cinder-849846-7-iscsi devstack@keystone.service[80412]: spawned uWSGI worker 2 (pid: 80424, cores: 1) Aug 06 21:49:10.668439 dv-cinder-849846-7-iscsi devstack@keystone.service[80412]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 06 21:49:16.231987 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 06 21:49:16.231987 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: from cryptography.utils import int_from_bytes Aug 06 21:49:16.233066 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 06 21:49:16.233066 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: from cryptography.utils import int_from_bytes Aug 06 21:49:16.233268 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 06 21:49:16.233268 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: from cryptography.utils import int_from_bytes Aug 06 21:49:16.233735 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 06 21:49:16.233735 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: from cryptography.utils import int_from_bytes Aug 06 21:49:16.282671 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 06 21:49:16.282858 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 06 21:49:16.283048 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] command line args: [] {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 06 21:49:16.283293 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 06 21:49:16.283386 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 06 21:49:16.283467 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] ================================================================================ {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 06 21:49:16.283547 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 06 21:49:16.283668 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] command line args: [] {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 06 21:49:16.283755 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] admin_token = **** {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.283940 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 06 21:49:16.284023 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] config_dir = [] {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.284121 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ================================================================================ {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 06 21:49:16.284201 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.284300 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] config_source = [] {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.284470 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] admin_token = **** {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.284569 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] debug = True {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.284699 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.285009 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] config_dir = [] {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.285127 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.285231 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.285337 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] config_source = [] {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.285440 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.285537 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] debug = True {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.285691 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.285833 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.285970 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.286083 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.286193 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.286295 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] list_limit = None {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.286414 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] log_config_append = None {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.286541 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.286658 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.286757 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] log_dir = None {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.286887 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] log_file = None {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.287001 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.287108 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.287185 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] list_limit = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.287362 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.287443 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] log_config_append = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.287597 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.287681 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.287800 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] log_dir = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.287876 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.288005 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] log_file = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.288089 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.288168 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.288287 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.288420 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.288663 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.288755 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.288902 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.289030 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.289136 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.289342 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.289436 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.289516 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.289590 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.289713 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.289824 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.290008 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.290096 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.290331 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.290454 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.290632 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.290821 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.18/identity {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.290945 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.291032 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] publish_errors = False {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.291227 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.291305 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.291461 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.291585 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.291661 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.291879 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.291959 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.292116 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.292259 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.292345 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] standard_threads = False {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.292593 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.292681 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.292837 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.293014 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.293189 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] use_journal = False {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.293364 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] use_json = False {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.293449 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.18/identity {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.293530 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] use_stderr = False {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.293610 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] publish_errors = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.293692 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] use_syslog = False {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.293887 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.293966 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.294165 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.294259 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.294396 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.294475 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.294654 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.294733 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.294892 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.295043 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.295145 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.295357 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] standard_threads = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.295442 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.295623 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.295708 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.295878 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.295997 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.296143 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.296223 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.296420 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] use_journal = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.296500 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.296632 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] use_json = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.296810 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.296897 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] use_stderr = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.297034 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] use_syslog = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.297118 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.297341 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 21:49:16.297433 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.297674 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.297754 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.297992 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.298073 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.298293 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.298373 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.298589 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.298671 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.298908 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.298994 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.299214 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.299332 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.299536 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.299680 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.299848 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.300021 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.300200 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.300476 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.300604 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.301311 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.301479 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.301571 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.301798 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.301878 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.302052 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.302131 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.302300 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.302386 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.302555 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.302694 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.302811 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.303006 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.303085 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.303303 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.303458 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.303566 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.303654 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.303815 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.303938 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.304054 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.304242 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.304329 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.304595 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.304699 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.304855 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.304943 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.305109 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.305191 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.305346 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.305432 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.305581 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.305667 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.305817 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.305898 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.306056 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.306142 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.306287 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.306366 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.306521 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.306602 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.306754 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.306834 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.306988 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.307107 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.307215 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.307313 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.307465 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.307569 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.307698 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.307808 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.307907 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.308010 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.308143 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.308224 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.308379 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.308536 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.308637 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.308718 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.308870 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.308950 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.309105 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.309197 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.309340 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.309428 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.309574 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.309654 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.309809 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.309894 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.310044 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.310133 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.310282 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.310363 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.310513 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.310593 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.310746 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.310826 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.310985 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.311065 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.311218 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.311298 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.311450 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.311540 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.311688 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.311769 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.311927 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.312006 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.312156 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.312243 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.312394 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.312538 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.312648 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.312730 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.312887 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.312977 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.313123 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.313202 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.313376 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.313464 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.313617 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.313698 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.313848 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.313927 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.314080 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.314170 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.314310 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.314397 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.314557 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.314637 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.314788 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.314866 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.315022 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.315182 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.315260 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.315331 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.315489 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.315574 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.315724 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.315813 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.315958 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.316038 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.316192 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.316271 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.317022 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.317279 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.317389 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.317519 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.317616 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.317770 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.317855 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.318010 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.318101 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.318250 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.318331 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.318495 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.318576 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.318735 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.318822 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.318971 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.319050 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.319207 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.319292 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.319440 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.319530 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.319671 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.319758 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.319903 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.319983 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.320138 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.320227 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.320369 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.320515 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.320632 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.320714 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.320869 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.320962 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.321108 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.321187 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.321344 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.321428 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.321580 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.321672 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.321813 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.321910 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.322048 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.322128 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.322285 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.322374 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] auth.external = None {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.322519 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.322600 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.322753 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.322833 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.322996 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.323085 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.323234 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.323315 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] auth.password = None {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.323469 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] auth.token = None {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.323547 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.323709 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.323797 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.323945 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.324056 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.324199 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.324290 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.324448 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.324547 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.324700 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.324790 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] auth.external = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.324932 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.325012 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.325168 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.325247 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.325402 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] credential.caching = True {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.325492 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.325650 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.325737 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] auth.password = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.325887 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] auth.token = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.325967 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.326129 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.326212 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.326364 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.326443 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.326607 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.326688 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.326837 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.326918 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.327086 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.327182 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.327336 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.327424 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.327568 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.327659 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.327806 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] credential.caching = True {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.327887 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.328055 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.328137 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.328307 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.328390 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.328584 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.328665 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.328824 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.328903 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.329069 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.329151 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.329302 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] federation.caching = True {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.329381 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.329538 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.329618 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.329767 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.329845 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.330019 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.330099 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.330249 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.330336 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.330495 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.330578 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.330730 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.330826 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.330984 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.331075 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.331219 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.331302 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.331471 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.331552 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.331707 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.331788 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] federation.caching = True {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.331974 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.332107 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.332202 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] identity.caching = True {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.332276 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.332468 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.332561 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.332709 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.332793 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.332943 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.333037 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.333178 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.333262 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.333425 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.333508 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.333656 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.333737 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.333894 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.333974 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.334136 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.334217 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.334385 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.334466 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.334613 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.334693 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] identity.caching = True {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.334844 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.334930 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.335074 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.335155 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.335316 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.335397 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.335550 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.335632 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.335797 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.335877 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.336029 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.336117 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.336269 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.336351 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.336548 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.336632 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.336784 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.336876 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.337020 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.337103 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.337250 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.337329 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.337484 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.337574 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.337712 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.337793 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.337946 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.338026 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.338184 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.338274 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.338423 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.338504 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.338670 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.338755 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.338900 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.338990 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.339147 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.339229 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.339389 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.339468 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.339621 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.339713 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.339857 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.339937 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.340090 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.340171 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.340327 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.340448 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.340587 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.340668 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.340818 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.340898 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.341056 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.341149 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.341287 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.341366 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.341526 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.341647 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.341767 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.341885 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.342001 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.342083 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.342252 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.342330 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.342497 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.342585 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.342729 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.342817 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.342958 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.343036 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.343187 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.343274 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.343426 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.343505 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.343663 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.343750 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.343898 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.343985 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.344132 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.344211 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.344360 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] ldap.user = None {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.344458 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.344633 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.344724 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.344884 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.344965 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.345114 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.345192 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.345345 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.345432 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.345593 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.345674 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.345836 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.345919 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.346072 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.346160 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.346300 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.346378 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.346534 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.346612 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.346766 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.346852 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.user = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.347000 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.347080 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.347230 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.347309 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.347477 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.348101 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.348235 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.348316 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.348503 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.348584 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.348753 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.348848 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.348991 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.349071 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.349218 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.349296 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.349464 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.349595 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.349708 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.349836 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.349943 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.350076 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.350163 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.350307 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.350387 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.350555 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.350633 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.350803 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.350891 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.351049 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.351646 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.351711 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.351771 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.351840 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.351902 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.351959 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.352024 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.352083 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.352148 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.352215 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.352276 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.352487 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.352565 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.352691 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.352766 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.352908 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.352972 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.353150 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.353237 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.353386 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.353506 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.353622 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.353702 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] resource.caching = True {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.353874 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.353952 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.354105 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.354182 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.354346 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.354426 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.354580 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.354658 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.354817 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.354896 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.355050 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.355129 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.355296 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.355375 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.355532 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.355610 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.355764 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.355843 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.355995 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] role.caching = True {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.356073 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.356232 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] resource.caching = True {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.356310 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] role.driver = sql {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.356481 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.356560 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.356733 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.356819 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.356977 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.357058 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.357226 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.357623 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.357733 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.357809 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.357972 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.358051 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.358216 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.358295 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.358451 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.358528 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.358690 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.358773 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.358919 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.358982 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] role.caching = True {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.359158 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.359237 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] role.driver = sql {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.359388 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.359466 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.359639 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.359718 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.359885 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.359991 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.360129 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.360301 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.360437 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.360623 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.360754 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.360854 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.360988 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.361105 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.361211 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.361366 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.361449 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.361602 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.361683 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.361853 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.361934 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.362101 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.362181 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.362349 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.362429 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.362587 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.362669 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.362823 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.363001 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.363081 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.363308 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.363994 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.364253 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.364334 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.364536 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.364618 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.364793 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.364890 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.365031 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.365110 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.365269 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.365349 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.365504 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.365655 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.365737 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] token.caching = True {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.365913 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.365993 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.366156 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.366235 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.366391 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.366473 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.366627 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.366707 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.366863 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.366942 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.367118 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.367199 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.367362 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.367439 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.367606 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.367686 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.367850 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.367931 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.368087 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.368165 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.368316 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.368394 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.368572 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.368652 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] token.caching = True {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.368808 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.368927 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.369057 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.369144 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.369288 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.369368 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.369529 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.369608 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.369780 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.369866 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.370020 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.370100 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] database.connection = **** {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.370266 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.370354 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.370510 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.370590 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.370757 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.370844 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.370994 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.371074 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.371226 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.371304 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.371459 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.371545 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.371690 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.371769 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.371925 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.372005 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.372161 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.372283 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.372393 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.372489 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.372654 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.372734 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.372887 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.372979 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] database.connection = **** {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.373137 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.373379 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.373460 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.373533 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.373606 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.373678 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.373819 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.373901 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.374063 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.374143 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.374298 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.374378 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.374459 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 06 21:49:16.374559 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.374797 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.375043 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.375280 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.375524 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.375762 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.376011 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.376247 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.376516 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.376752 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.376993 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.377230 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 21:49:16.377399 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 06 21:49:16.412903 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 21:49:16.413132 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 21:49:16.413218 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 21:49:16.413353 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 21:49:16.413460 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 21:49:16.413635 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 21:49:16.423054 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 21:49:16.423361 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 21:49:16.451640 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 21:49:16.451870 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 21:49:16.452092 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 21:49:16.452326 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 21:49:16.504722 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 21:49:16.504939 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 21:49:16.555833 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 21:49:16.556066 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 21:49:16.556806 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 21:49:16.557045 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 21:49:16.621309 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 21:49:16.621484 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 21:49:16.621574 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 21:49:16.621711 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 21:49:16.621800 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 21:49:16.621930 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 21:49:16.622027 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 21:49:16.622142 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 21:49:16.622228 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 21:49:16.622353 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 21:49:16.622437 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 21:49:16.622568 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 21:49:16.645479 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 21:49:16.645648 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 21:49:16.645738 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 21:49:16.645897 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 21:49:16.645985 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 21:49:16.646116 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 21:49:16.646201 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 21:49:16.646332 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 21:49:16.646415 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 21:49:16.646547 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 21:49:16.646639 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 21:49:16.646761 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 21:49:16.646845 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 21:49:16.646975 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 21:49:16.657837 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 21:49:16.658176 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 21:49:16.670790 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 21:49:16.671018 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 21:49:16.671236 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 21:49:16.671366 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 21:49:16.671494 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 21:49:16.671648 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 21:49:16.671777 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 21:49:16.671901 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 21:49:16.672028 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 21:49:16.672153 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 21:49:16.672285 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 21:49:16.672544 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 21:49:16.672544 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 21:49:16.672544 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 21:49:16.689397 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.689607 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 06 21:49:16.689819 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.689949 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.690078 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 06 21:49:16.690206 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 06 21:49:16.690364 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.690500 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.690630 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 06 21:49:16.690758 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.690887 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.691015 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.691139 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.691312 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.691439 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.691601 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.702514 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.702793 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.702997 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.703124 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.703275 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.703400 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.703532 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.703816 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.710341 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=80423) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 21:49:16.710760 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=80424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 21:49:16.713395 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=80423) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 21:49:16.713683 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.713869 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=80424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 21:49:16.714011 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.714164 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.714291 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.714449 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.714574 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.714733 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.714870 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.714996 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.715119 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.715268 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.715409 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.715536 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.715659 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.715784 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.715908 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.716053 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.716186 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.716322 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.716612 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.740116 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.740953 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.741617 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80423) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 21:49:16.741902 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.742447 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 21:49:16.742736 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.746494 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=80423) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 21:49:16.746777 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.747057 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.747339 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=80424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 21:49:16.747623 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.747902 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.753635 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=80423) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 21:49:16.753948 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.754631 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=80424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 21:49:16.754960 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.758030 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80423) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 21:49:16.758324 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.758609 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.758900 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.759026 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 21:49:16.759176 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.759303 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.759466 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.759593 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.759741 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.759868 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.760022 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.760149 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.760395 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.760697 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.760990 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.776337 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.776640 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.777491 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.777774 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.780579 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80423) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 21:49:16.781773 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 21:49:16.783707 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.784002 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.784948 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.785802 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.789079 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80423) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 21:49:16.790199 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 21:49:16.792216 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80423) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 21:49:16.793388 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 21:49:16.795384 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.795690 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.795980 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.796265 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.796567 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.796698 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.796856 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.796982 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.797155 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.797443 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.797724 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.798005 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.812767 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80423) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 21:49:16.813055 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.813330 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.813595 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.813974 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 21:49:16.814266 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.814546 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.814814 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.820524 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.821743 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.822069 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.822347 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.823313 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.823594 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.825183 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=80423) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 21:49:16.825468 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.825776 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.826067 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.826346 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.826471 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=80424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 21:49:16.826700 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.827001 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.827296 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.827577 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.838201 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=80423) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 21:49:16.838486 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.838770 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.839059 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.839336 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.839613 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.839794 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=80424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 21:49:16.839881 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.840083 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.840367 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.840675 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.840959 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.841239 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.841517 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.857224 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=80423) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 21:49:16.859166 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=80424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 21:49:16.860441 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80423) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 21:49:16.862141 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 21:49:16.863483 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.865093 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.865263 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.866586 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=80423) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 21:49:16.866844 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.866928 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.867157 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.868334 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=80424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 21:49:16.868644 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.868946 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.873545 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.875141 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=80423) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 21:49:16.875366 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.876994 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=80424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 21:49:16.878101 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.878393 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.878680 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.878999 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.880051 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.880344 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.880659 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.880942 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.887175 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80423) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 21:49:16.887467 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.887752 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.889724 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 21:49:16.890021 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.890536 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.895468 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=80423) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 21:49:16.895753 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.896035 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.896311 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.896605 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.896884 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.897169 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.897446 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.897826 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.897955 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=80424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 21:49:16.898100 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.898242 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.898372 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.898522 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.898643 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.898800 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.898920 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.899074 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.899183 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.899346 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.899632 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.899901 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.900285 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.900599 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.900876 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.901141 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.901427 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.901692 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 21:49:16.933111 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55cbdb8299c0 pid: 80423 (default app) Aug 06 21:49:16.936319 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de2e2c1-1324-4244-83a5-7dc9dd6510ba None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:16.936624 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de2e2c1-1324-4244-83a5-7dc9dd6510ba None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:16.936904 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de2e2c1-1324-4244-83a5-7dc9dd6510ba None None] PATH_INFO: `/v3` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:16.941142 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1/1] 10.222.0.18 () {50 vars in 791 bytes} [Sat Aug 6 21:49:16 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:49:16.943390 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55cbdb8299c0 pid: 80424 (default app) Aug 06 21:49:25.890062 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e69cc-8621-4825-8a5e-4825e9162377 None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:25.890631 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e69cc-8621-4825-8a5e-4825e9162377 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:25.890631 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e69cc-8621-4825-8a5e-4825e9162377 None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:25.891918 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 2/2] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:49:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:49:25.900125 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f02e3d2-4abf-4c13-aef0-910eec7ce0c0 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:25.900585 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f02e3d2-4abf-4c13-aef0-910eec7ce0c0 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:25.900712 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f02e3d2-4abf-4c13-aef0-910eec7ce0c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:25.971346 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: INFO dbcounter [None req-7f02e3d2-4abf-4c13-aef0-910eec7ce0c0 None None] Registered counter for database keystone Aug 06 21:49:25.991101 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG oslo_db.sqlalchemy.engines [None req-7f02e3d2-4abf-4c13-aef0-910eec7ce0c0 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80424) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 06 21:49:25.997329 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writer thread running {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 06 21:49:26.175409 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG passlib.handlers.bcrypt [None req-7f02e3d2-4abf-4c13-aef0-910eec7ce0c0 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80424) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 06 21:49:26.175780 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG passlib.handlers.bcrypt [None req-7f02e3d2-4abf-4c13-aef0-910eec7ce0c0 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80424) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 06 21:49:26.232724 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-7f02e3d2-4abf-4c13-aef0-910eec7ce0c0 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:49:26.234389 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-7f02e3d2-4abf-4c13-aef0-910eec7ce0c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:49:26.369941 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1/3] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:49:25 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 473 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 06 21:49:26.500942 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0e69cc-8621-4825-8a5e-4825e9162377 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:26.575043 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: INFO dbcounter [None req-1c0e69cc-8621-4825-8a5e-4825e9162377 None None] Registered counter for database keystone Aug 06 21:49:26.596463 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG oslo_db.sqlalchemy.engines [None req-1c0e69cc-8621-4825-8a5e-4825e9162377 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80423) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 06 21:49:26.603099 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writer thread running {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 06 21:49:26.723307 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0e69cc-8621-4825-8a5e-4825e9162377 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:26.727977 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a843085d-5278-403c-83ff-dd7184db8cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:26.728702 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a843085d-5278-403c-83ff-dd7184db8cfa admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:26.728988 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a843085d-5278-403c-83ff-dd7184db8cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:26.729266 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a843085d-5278-403c-83ff-dd7184db8cfa admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:26.737580 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a843085d-5278-403c-83ff-dd7184db8cfa admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:26.791335 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: WARNING keystone.common.rbac_enforcer.enforcer [None req-a843085d-5278-403c-83ff-dd7184db8cfa admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 06 21:49:26.791619 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a843085d-5278-403c-83ff-dd7184db8cfa admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:26.801310 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 3/4] 10.222.0.18 () {58 vars in 1324 bytes} [Sat Aug 6 21:49:26 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 21:49:26.807774 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f02e3d2-4abf-4c13-aef0-910eec7ce0c0 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:26.822716 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f02e3d2-4abf-4c13-aef0-910eec7ce0c0 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:26.827746 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf378b6-d93e-4c90-8959-533d8fd8eb26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:26.828277 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf378b6-d93e-4c90-8959-533d8fd8eb26 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:26.828596 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf378b6-d93e-4c90-8959-533d8fd8eb26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:26.828881 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf378b6-d93e-4c90-8959-533d8fd8eb26 admin admin] PATH_INFO: `/v3/projects/c285ce4925f04fb5acce87c789a71e83` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:26.830308 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf378b6-d93e-4c90-8959-533d8fd8eb26 admin admin] RBAC: Authorizing `identity:get_project(project_id=c285ce4925f04fb5acce87c789a71e83)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:26.882588 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: WARNING keystone.common.rbac_enforcer.enforcer [None req-9cf378b6-d93e-4c90-8959-533d8fd8eb26 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 06 21:49:26.882931 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf378b6-d93e-4c90-8959-533d8fd8eb26 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:26.887450 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 2/5] 10.222.0.18 () {56 vars in 1207 bytes} [Sat Aug 6 21:49:26 2022] GET /identity/v3/projects/c285ce4925f04fb5acce87c789a71e83 => generated 344 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:49:28.852302 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f15b15-5ece-4f65-b16c-28126eacb899 None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:28.852858 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f15b15-5ece-4f65-b16c-28126eacb899 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:28.852858 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f15b15-5ece-4f65-b16c-28126eacb899 None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:28.856961 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 4/6] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:49:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:49:28.862651 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2010f68c-68b2-44a7-8fcd-2b518ace4113 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:28.863022 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2010f68c-68b2-44a7-8fcd-2b518ace4113 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:28.863208 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2010f68c-68b2-44a7-8fcd-2b518ace4113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:28.885601 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-2010f68c-68b2-44a7-8fcd-2b518ace4113 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:49:28.887121 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-2010f68c-68b2-44a7-8fcd-2b518ace4113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:49:28.894941 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 3/7] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:49:28 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 06 21:49:28.916708 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f15b15-5ece-4f65-b16c-28126eacb899 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:28.929429 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f15b15-5ece-4f65-b16c-28126eacb899 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:28.933676 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b571ef52-3cd2-42a7-8c8c-8514c02b3ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:28.934264 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b571ef52-3cd2-42a7-8c8c-8514c02b3ce1 admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:28.934545 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b571ef52-3cd2-42a7-8c8c-8514c02b3ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:28.934819 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b571ef52-3cd2-42a7-8c8c-8514c02b3ce1 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:28.943007 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b571ef52-3cd2-42a7-8c8c-8514c02b3ce1 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:28.944803 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b571ef52-3cd2-42a7-8c8c-8514c02b3ce1 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:28.953400 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 5/8] 10.222.0.18 () {58 vars in 1324 bytes} [Sat Aug 6 21:49:28 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 21:49:28.958891 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2010f68c-68b2-44a7-8fcd-2b518ace4113 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:28.967772 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2010f68c-68b2-44a7-8fcd-2b518ace4113 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:28.972073 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34248891-9a75-4599-8ab4-103fd1289830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:28.972724 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34248891-9a75-4599-8ab4-103fd1289830 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:28.973001 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34248891-9a75-4599-8ab4-103fd1289830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:28.973269 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34248891-9a75-4599-8ab4-103fd1289830 admin admin] PATH_INFO: `/v3/users/6bb7ac25afcb4fcaba91aca891ca7fc9` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:28.974577 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34248891-9a75-4599-8ab4-103fd1289830 admin admin] RBAC: Authorizing `identity:get_user(user_id=6bb7ac25afcb4fcaba91aca891ca7fc9)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:28.976360 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34248891-9a75-4599-8ab4-103fd1289830 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:28.977895 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 4/9] 10.222.0.18 () {56 vars in 1198 bytes} [Sat Aug 6 21:49:28 2022] GET /identity/v3/users/6bb7ac25afcb4fcaba91aca891ca7fc9 => generated 247 bytes in 19 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:49:31.035039 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec8b8a7-52c4-413e-9516-efe8ab1b1893 None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:31.035579 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec8b8a7-52c4-413e-9516-efe8ab1b1893 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:31.035579 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec8b8a7-52c4-413e-9516-efe8ab1b1893 None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:31.037492 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 6/10] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:49:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:49:31.042521 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780ef840-875d-47ee-925b-11dd938f3394 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:31.042909 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780ef840-875d-47ee-925b-11dd938f3394 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:31.043201 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780ef840-875d-47ee-925b-11dd938f3394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:31.060952 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-780ef840-875d-47ee-925b-11dd938f3394 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:49:31.062134 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-780ef840-875d-47ee-925b-11dd938f3394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:49:31.082030 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c0cbb8-8310-428d-ae5b-f605ccd6365f None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:31.082313 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c0cbb8-8310-428d-ae5b-f605ccd6365f None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:31.083155 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c0cbb8-8310-428d-ae5b-f605ccd6365f None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:31.084715 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 7/11] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:49:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 21:49:31.089734 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011f5637-b396-45fb-8272-a21091912765 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:31.090084 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011f5637-b396-45fb-8272-a21091912765 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:31.090437 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011f5637-b396-45fb-8272-a21091912765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:31.133717 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG passlib.handlers.bcrypt [None req-011f5637-b396-45fb-8272-a21091912765 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80423) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 06 21:49:31.134397 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG passlib.handlers.bcrypt [None req-011f5637-b396-45fb-8272-a21091912765 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80423) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 06 21:49:31.143725 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 5/12] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:49:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 21:49:31.170926 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780ef840-875d-47ee-925b-11dd938f3394 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:31.182080 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-011f5637-b396-45fb-8272-a21091912765 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:49:31.183719 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-011f5637-b396-45fb-8272-a21091912765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:49:31.194601 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:31.274327 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780ef840-875d-47ee-925b-11dd938f3394 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:31.282587 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 8/13] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:49:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 21:49:31.304688 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011f5637-b396-45fb-8272-a21091912765 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:31.309144 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d64d953-412f-40dd-8595-8a7087de4626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:31.309780 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d64d953-412f-40dd-8595-8a7087de4626 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:31.310062 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d64d953-412f-40dd-8595-8a7087de4626 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:31.310335 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d64d953-412f-40dd-8595-8a7087de4626 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:31.321938 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d64d953-412f-40dd-8595-8a7087de4626 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:49:31.330651 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-1d64d953-412f-40dd-8595-8a7087de4626 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 21:49:31.330651 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:49:31.330651 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:49:31.330651 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:49:31.330651 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:49:31.330651 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:49:31.330651 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:49:31.330651 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:49:31.330651 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:49:31.330651 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:49:31.330651 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:49:31.330651 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:49:31.330651 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:49:31.330651 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:49:31.330651 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:49:31.330651 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:49:31.331551 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:49:31.331551 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:49:31.331551 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:49:31.331551 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:49:31.331551 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:49:31.331551 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:49:31.331551 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:49:31.331551 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:49:31.331551 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:49:31.331551 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:49:31.331551 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:49:31.331551 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:49:31.331551 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:49:31.331551 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:49:31.331551 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:49:31.331551 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:49:31.332379 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:49:31.332379 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:49:31.332379 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:49:31.332379 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:49:31.332379 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:49:31.332379 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:49:31.332379 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:49:31.332379 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:49:31.332379 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:49:31.332379 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:49:31.332379 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 21:49:31.332379 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:49:31.332379 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 6/14] 10.222.0.18 () {56 vars in 1096 bytes} [Sat Aug 6 21:49:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 21:49:31.338144 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d64d953-412f-40dd-8595-8a7087de4626 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:31.410861 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011f5637-b396-45fb-8272-a21091912765 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:31.413348 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:31.418853 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:31.436755 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d64d953-412f-40dd-8595-8a7087de4626 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:31.447098 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03059c7c-eab4-4e73-803a-a16aa7d789af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:31.447955 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03059c7c-eab4-4e73-803a-a16aa7d789af None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:31.448236 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03059c7c-eab4-4e73-803a-a16aa7d789af None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:31.448548 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03059c7c-eab4-4e73-803a-a16aa7d789af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:31.470215 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df88a1e7-6172-4cfd-824f-fb68678e5a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:31.470848 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df88a1e7-6172-4cfd-824f-fb68678e5a16 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:31.471180 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df88a1e7-6172-4cfd-824f-fb68678e5a16 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:31.471456 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df88a1e7-6172-4cfd-824f-fb68678e5a16 None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:31.472124 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df88a1e7-6172-4cfd-824f-fb68678e5a16 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:49:31.472445 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df88a1e7-6172-4cfd-824f-fb68678e5a16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:31.474035 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df88a1e7-6172-4cfd-824f-fb68678e5a16 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:31.485847 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 7/15] 10.222.0.18 () {56 vars in 1099 bytes} [Sat Aug 6 21:49:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:49:31.491621 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df88a1e7-6172-4cfd-824f-fb68678e5a16 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:31.515166 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03059c7c-eab4-4e73-803a-a16aa7d789af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:31.516743 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03059c7c-eab4-4e73-803a-a16aa7d789af None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:31.622112 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 9/16] 10.222.0.18 () {58 vars in 1282 bytes} [Sat Aug 6 21:49:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 06 21:49:31.627873 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03059c7c-eab4-4e73-803a-a16aa7d789af None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:31.631311 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:31.700423 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df88a1e7-6172-4cfd-824f-fb68678e5a16 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:31.734811 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03059c7c-eab4-4e73-803a-a16aa7d789af None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:31.736679 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2320acb-a7da-4203-85d2-0eff18ce98bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:31.737309 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2320acb-a7da-4203-85d2-0eff18ce98bb None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:31.737589 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2320acb-a7da-4203-85d2-0eff18ce98bb None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:31.737868 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2320acb-a7da-4203-85d2-0eff18ce98bb None admin] PATH_INFO: `/v3/users/6bb7ac25afcb4fcaba91aca891ca7fc9` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:31.739098 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2320acb-a7da-4203-85d2-0eff18ce98bb None admin] RBAC: Authorizing `identity:get_user(user_id=6bb7ac25afcb4fcaba91aca891ca7fc9)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:31.740605 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2320acb-a7da-4203-85d2-0eff18ce98bb None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:31.742163 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 8/17] 10.222.0.18 () {56 vars in 1177 bytes} [Sat Aug 6 21:49:31 2022] GET /identity/v3/users/6bb7ac25afcb4fcaba91aca891ca7fc9 => generated 247 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:49:31.747508 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2320acb-a7da-4203-85d2-0eff18ce98bb None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:31.761901 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:31.767760 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0583bc40-9c6a-4470-8c49-1b56135531ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:31.768652 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0583bc40-9c6a-4470-8c49-1b56135531ad None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:31.768928 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0583bc40-9c6a-4470-8c49-1b56135531ad None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:31.769197 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0583bc40-9c6a-4470-8c49-1b56135531ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:31.788751 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0583bc40-9c6a-4470-8c49-1b56135531ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:31.790229 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0583bc40-9c6a-4470-8c49-1b56135531ad None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:31.808278 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-0583bc40-9c6a-4470-8c49-1b56135531ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:49:31.808278 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:49:31.808278 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:49:31.808278 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:49:31.808278 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:49:31.808278 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:49:31.808278 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:49:31.808278 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:49:31.808278 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:49:31.808278 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:49:31.808278 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:49:31.808278 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:49:31.808278 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:49:31.808278 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:49:31.808278 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:49:31.808278 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:49:31.809203 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:49:31.809203 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:49:31.809203 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:49:31.809203 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:49:31.809203 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:49:31.809203 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:49:31.809203 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:49:31.809203 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:49:31.809203 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:49:31.809203 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:49:31.809203 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:49:31.809203 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:49:31.809203 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:49:31.809203 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:49:31.809203 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:49:31.809203 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:49:31.809203 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:49:31.810264 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:49:31.810264 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:49:31.810264 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:49:31.810264 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 10/18] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:49:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 21:49:31.815075 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0583bc40-9c6a-4470-8c49-1b56135531ad None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:31.845526 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2320acb-a7da-4203-85d2-0eff18ce98bb None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:31.872295 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:31.880169 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553fed11-f191-42b2-94c3-016f27e9f37c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:31.886710 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553fed11-f191-42b2-94c3-016f27e9f37c None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:31.886710 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553fed11-f191-42b2-94c3-016f27e9f37c None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:31.886710 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553fed11-f191-42b2-94c3-016f27e9f37c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:31.886710 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553fed11-f191-42b2-94c3-016f27e9f37c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:31.886710 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553fed11-f191-42b2-94c3-016f27e9f37c None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:31.886710 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 9/19] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:49:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:49:31.891361 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553fed11-f191-42b2-94c3-016f27e9f37c None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:31.916756 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0583bc40-9c6a-4470-8c49-1b56135531ad None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:31.950823 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61bb09f-feec-45d8-9f46-00beb3fba758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:31.951430 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61bb09f-feec-45d8-9f46-00beb3fba758 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:31.951705 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61bb09f-feec-45d8-9f46-00beb3fba758 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:31.951977 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61bb09f-feec-45d8-9f46-00beb3fba758 None admin] PATH_INFO: `/v3/domains` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:31.952646 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61bb09f-feec-45d8-9f46-00beb3fba758 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:49:31.952947 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61bb09f-feec-45d8-9f46-00beb3fba758 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:31.954338 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61bb09f-feec-45d8-9f46-00beb3fba758 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:31.971996 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 11/20] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:49:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:49:31.976833 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:31.997658 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553fed11-f191-42b2-94c3-016f27e9f37c None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:32.036427 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9407981a-9bd1-45ae-aedf-fa0566a08a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:32.037259 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9407981a-9bd1-45ae-aedf-fa0566a08a1d None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:32.037561 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9407981a-9bd1-45ae-aedf-fa0566a08a1d None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:32.037837 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9407981a-9bd1-45ae-aedf-fa0566a08a1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:32.038581 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9407981a-9bd1-45ae-aedf-fa0566a08a1d None admin] RBAC: Adding query filter params (role.id=4b8dcf0491fd4072acc69fd894bf77d0, scope.domain.id=default, user.id=6bb7ac25afcb4fcaba91aca891ca7fc9) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:49:32.038887 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9407981a-9bd1-45ae-aedf-fa0566a08a1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:32.040390 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9407981a-9bd1-45ae-aedf-fa0566a08a1d None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:32.048128 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 10/21] 10.222.0.18 () {56 vars in 1322 bytes} [Sat Aug 6 21:49:31 2022] GET /identity/v3/role_assignments?user.id=6bb7ac25afcb4fcaba91aca891ca7fc9&scope.domain.id=default&role.id=4b8dcf0491fd4072acc69fd894bf77d0 => generated 233 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:49:34.289496 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ff3cf-f437-4e81-931c-c33637e71167 None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:34.290539 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ff3cf-f437-4e81-931c-c33637e71167 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:34.291025 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ff3cf-f437-4e81-931c-c33637e71167 None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:34.292665 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 12/22] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:49:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:49:34.297789 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ea444-2b1c-4158-bb51-23adca3f571b None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:34.298599 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ea444-2b1c-4158-bb51-23adca3f571b None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:34.299090 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ea444-2b1c-4158-bb51-23adca3f571b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:34.322751 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-c79ea444-2b1c-4158-bb51-23adca3f571b None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:49:34.324300 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-c79ea444-2b1c-4158-bb51-23adca3f571b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:49:34.416915 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 11/23] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:49:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 21:49:34.451253 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007ff3cf-f437-4e81-931c-c33637e71167 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:34.583293 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007ff3cf-f437-4e81-931c-c33637e71167 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:34.605759 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:34.631204 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7354ee2e-de41-4a64-a1d2-08c591d69a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:34.631853 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7354ee2e-de41-4a64-a1d2-08c591d69a7a None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:34.633731 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7354ee2e-de41-4a64-a1d2-08c591d69a7a None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:34.633731 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7354ee2e-de41-4a64-a1d2-08c591d69a7a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:34.649612 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7354ee2e-de41-4a64-a1d2-08c591d69a7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:49:34.730675 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcca2f65-8971-48e8-9373-aca6c4cacc17 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:34.731305 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcca2f65-8971-48e8-9373-aca6c4cacc17 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:34.731809 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcca2f65-8971-48e8-9373-aca6c4cacc17 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:34.737838 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 12/24] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:49:34 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:49:34.744536 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1b7d94-852e-450f-a195-c78c4acae69c None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:34.745142 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1b7d94-852e-450f-a195-c78c4acae69c None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:34.745622 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1b7d94-852e-450f-a195-c78c4acae69c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:34.769526 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-7354ee2e-de41-4a64-a1d2-08c591d69a7a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 21:49:34.769526 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:49:34.769526 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:49:34.769526 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:49:34.769526 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:49:34.769526 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:49:34.769526 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:49:34.769526 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:49:34.769526 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:49:34.769526 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:49:34.769526 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:49:34.769526 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:49:34.769526 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:49:34.769526 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:49:34.769526 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:49:34.769526 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:49:34.770918 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:49:34.770918 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:49:34.770918 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:49:34.770918 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:49:34.770918 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:49:34.770918 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:49:34.770918 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:49:34.770918 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:49:34.770918 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:49:34.770918 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:49:34.770918 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:49:34.770918 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:49:34.770918 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:49:34.770918 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:49:34.770918 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:49:34.770918 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:49:34.771692 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:49:34.771692 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:49:34.771692 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:49:34.771692 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:49:34.771692 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:49:34.771692 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:49:34.771692 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:49:34.771692 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:49:34.771692 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:49:34.771692 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:49:34.771692 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 21:49:34.771692 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:49:34.771692 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 13/25] 10.222.0.18 () {56 vars in 1096 bytes} [Sat Aug 6 21:49:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:49:34.779062 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7354ee2e-de41-4a64-a1d2-08c591d69a7a None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:34.781201 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-ca1b7d94-852e-450f-a195-c78c4acae69c None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:49:34.782399 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-ca1b7d94-852e-450f-a195-c78c4acae69c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:49:34.958856 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 13/26] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:49:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 21:49:34.973997 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1214771-7cb1-4fb9-b192-8e6b61eedf82 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:34.974296 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1214771-7cb1-4fb9-b192-8e6b61eedf82 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:34.974549 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1214771-7cb1-4fb9-b192-8e6b61eedf82 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:34.975886 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 14/27] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:49:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 21:49:34.987333 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c03bb47-ee5f-4255-8fd9-800484c63238 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:34.987618 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c03bb47-ee5f-4255-8fd9-800484c63238 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:34.987895 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c03bb47-ee5f-4255-8fd9-800484c63238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:34.997662 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:35.008784 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7354ee2e-de41-4a64-a1d2-08c591d69a7a None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:35.013911 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-2c03bb47-ee5f-4255-8fd9-800484c63238 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:49:35.013911 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-2c03bb47-ee5f-4255-8fd9-800484c63238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:49:35.071953 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ab4f65-4fe4-468c-8690-05fc5f134f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:35.072757 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ab4f65-4fe4-468c-8690-05fc5f134f87 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:35.073040 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ab4f65-4fe4-468c-8690-05fc5f134f87 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:35.073361 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ab4f65-4fe4-468c-8690-05fc5f134f87 None admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:35.074050 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ab4f65-4fe4-468c-8690-05fc5f134f87 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:49:35.074451 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ab4f65-4fe4-468c-8690-05fc5f134f87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:35.076004 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ab4f65-4fe4-468c-8690-05fc5f134f87 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:35.083043 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 15/28] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:49:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 21:49:35.083720 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c03bb47-ee5f-4255-8fd9-800484c63238 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:35.099882 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 14/29] 10.222.0.18 () {56 vars in 1099 bytes} [Sat Aug 6 21:49:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:49:35.101745 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07de77df-fcc0-44df-92af-56b9d3239c4b None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:35.102219 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07de77df-fcc0-44df-92af-56b9d3239c4b None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:35.102677 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07de77df-fcc0-44df-92af-56b9d3239c4b None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:35.104144 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 15/30] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:49:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:49:35.110549 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07de77df-fcc0-44df-92af-56b9d3239c4b None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:35.184783 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:35.220260 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c03bb47-ee5f-4255-8fd9-800484c63238 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:35.246297 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07de77df-fcc0-44df-92af-56b9d3239c4b None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:35.256727 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99d6e4e-722a-488e-9363-d92584731c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:35.257545 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99d6e4e-722a-488e-9363-d92584731c49 None admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:35.257975 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99d6e4e-722a-488e-9363-d92584731c49 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:35.258395 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99d6e4e-722a-488e-9363-d92584731c49 None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:35.259191 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99d6e4e-722a-488e-9363-d92584731c49 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:35.260802 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99d6e4e-722a-488e-9363-d92584731c49 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:35.264856 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:35.284948 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66b95c2-9d76-4932-82bf-2bcd0202423f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:35.285585 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66b95c2-9d76-4932-82bf-2bcd0202423f None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:35.285861 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66b95c2-9d76-4932-82bf-2bcd0202423f None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:35.286126 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66b95c2-9d76-4932-82bf-2bcd0202423f None admin] PATH_INFO: `/v3/users/6bb7ac25afcb4fcaba91aca891ca7fc9` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:35.287685 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66b95c2-9d76-4932-82bf-2bcd0202423f None admin] RBAC: Authorizing `identity:get_user(user_id=6bb7ac25afcb4fcaba91aca891ca7fc9)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:35.289206 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66b95c2-9d76-4932-82bf-2bcd0202423f None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:35.291010 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 16/31] 10.222.0.18 () {56 vars in 1177 bytes} [Sat Aug 6 21:49:35 2022] GET /identity/v3/users/6bb7ac25afcb4fcaba91aca891ca7fc9 => generated 247 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:49:35.292033 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f46d8a-476b-43b8-a96c-83f37a226247 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:35.292316 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f46d8a-476b-43b8-a96c-83f37a226247 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:35.292612 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f46d8a-476b-43b8-a96c-83f37a226247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:35.311409 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-a1f46d8a-476b-43b8-a96c-83f37a226247 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:49:35.312582 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-a1f46d8a-476b-43b8-a96c-83f37a226247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:49:35.394128 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 17/32] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:49:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 21:49:35.395728 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f46d8a-476b-43b8-a96c-83f37a226247 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:35.441418 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:35.497393 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 16/33] 10.222.0.18 () {60 vars in 1131 bytes} [Sat Aug 6 21:49:35 2022] POST /identity/v3/roles => generated 219 bytes in 414 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 21:49:35.498442 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99d6e4e-722a-488e-9363-d92584731c49 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:35.526816 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f46d8a-476b-43b8-a96c-83f37a226247 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:35.581618 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5495601-adc2-4180-a8ea-580db2f70b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:35.583126 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5495601-adc2-4180-a8ea-580db2f70b4a None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:35.583918 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5495601-adc2-4180-a8ea-580db2f70b4a None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:35.584372 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5495601-adc2-4180-a8ea-580db2f70b4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:35.599835 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5495601-adc2-4180-a8ea-580db2f70b4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:35.602113 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5495601-adc2-4180-a8ea-580db2f70b4a None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:35.620079 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-f5495601-adc2-4180-a8ea-580db2f70b4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:49:35.620079 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:49:35.620079 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:49:35.620079 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:49:35.620079 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:49:35.620079 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:49:35.620079 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:49:35.620079 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:49:35.620079 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:49:35.620079 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:49:35.620079 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:49:35.620079 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:49:35.620079 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:49:35.620079 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:49:35.620079 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:49:35.620079 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:49:35.621210 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:49:35.621210 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:49:35.621210 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:49:35.621210 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:49:35.621210 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:49:35.621210 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:49:35.621210 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:49:35.621210 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:49:35.621210 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:49:35.621210 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:49:35.621210 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:49:35.621210 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:49:35.621210 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:49:35.621210 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:49:35.621210 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:49:35.621210 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:49:35.621210 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:49:35.622230 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:49:35.622230 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:49:35.622230 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:49:35.623790 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 18/34] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:49:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:49:35.625091 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5495601-adc2-4180-a8ea-580db2f70b4a None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:35.628547 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99d6e4e-722a-488e-9363-d92584731c49 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:35.678588 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35466a4f-b1aa-48c3-80d1-86f67623f9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:35.679421 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35466a4f-b1aa-48c3-80d1-86f67623f9f0 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:35.679706 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35466a4f-b1aa-48c3-80d1-86f67623f9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:35.679980 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35466a4f-b1aa-48c3-80d1-86f67623f9f0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:35.681280 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35466a4f-b1aa-48c3-80d1-86f67623f9f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:35.682866 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35466a4f-b1aa-48c3-80d1-86f67623f9f0 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:35.685581 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 17/35] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:49:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:49:35.686804 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af44d03-ce63-4c9d-ad4e-0097d0c80853 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:35.687092 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af44d03-ce63-4c9d-ad4e-0097d0c80853 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:35.687368 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af44d03-ce63-4c9d-ad4e-0097d0c80853 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:35.688917 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 18/36] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:49:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:49:35.689634 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af44d03-ce63-4c9d-ad4e-0097d0c80853 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:35.750928 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:35.766018 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:35.766867 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5495601-adc2-4180-a8ea-580db2f70b4a None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:35.806118 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664afd3b-7c62-48d1-86d3-2b2f4c90dc08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:35.806736 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664afd3b-7c62-48d1-86d3-2b2f4c90dc08 None admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:35.807109 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664afd3b-7c62-48d1-86d3-2b2f4c90dc08 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:35.807396 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664afd3b-7c62-48d1-86d3-2b2f4c90dc08 None admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:35.808068 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664afd3b-7c62-48d1-86d3-2b2f4c90dc08 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:35.809068 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af44d03-ce63-4c9d-ad4e-0097d0c80853 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:35.809470 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664afd3b-7c62-48d1-86d3-2b2f4c90dc08 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:35.843418 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb681c1-1148-4746-95ba-9751a48e7f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:35.844028 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb681c1-1148-4746-95ba-9751a48e7f08 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:35.844310 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb681c1-1148-4746-95ba-9751a48e7f08 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:35.844616 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb681c1-1148-4746-95ba-9751a48e7f08 None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:35.845262 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb681c1-1148-4746-95ba-9751a48e7f08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:49:35.845563 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb681c1-1148-4746-95ba-9751a48e7f08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:35.846951 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb681c1-1148-4746-95ba-9751a48e7f08 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:35.865344 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 19/37] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:49:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:49:35.867808 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb681c1-1148-4746-95ba-9751a48e7f08 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:35.967817 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb681c1-1148-4746-95ba-9751a48e7f08 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:35.991761 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:35.993451 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 19/38] 10.222.0.18 () {60 vars in 1131 bytes} [Sat Aug 6 21:49:35 2022] POST /identity/v3/roles => generated 225 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 21:49:35.994913 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ab847a-af92-4ab1-b91f-fb6f3660ba9f None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:35.995202 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ab847a-af92-4ab1-b91f-fb6f3660ba9f None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:35.995479 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ab847a-af92-4ab1-b91f-fb6f3660ba9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:36.023760 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf6dd71-f952-4d0b-9e6e-40b91c52edc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:36.024363 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf6dd71-f952-4d0b-9e6e-40b91c52edc0 None admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:36.024674 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf6dd71-f952-4d0b-9e6e-40b91c52edc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:36.024957 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf6dd71-f952-4d0b-9e6e-40b91c52edc0 None admin] PATH_INFO: `/v3/domains/default/users/6bb7ac25afcb4fcaba91aca891ca7fc9/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:36.025255 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-f6ab847a-af92-4ab1-b91f-fb6f3660ba9f None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:49:36.026943 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-f6ab847a-af92-4ab1-b91f-fb6f3660ba9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:49:36.027081 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf6dd71-f952-4d0b-9e6e-40b91c52edc0 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=6bb7ac25afcb4fcaba91aca891ca7fc9, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:36.028641 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf6dd71-f952-4d0b-9e6e-40b91c52edc0 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:36.106728 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 20/39] 10.222.0.18 () {58 vars in 1361 bytes} [Sat Aug 6 21:49:35 2022] PUT /identity/v3/domains/default/users/6bb7ac25afcb4fcaba91aca891ca7fc9/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 21:49:36.108607 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 20/40] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:49:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 21:49:36.109949 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f90f580-dcee-4af8-9ae8-d8a066929157 None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:36.110240 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f90f580-dcee-4af8-9ae8-d8a066929157 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:36.110521 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f90f580-dcee-4af8-9ae8-d8a066929157 None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:36.112451 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 21/41] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:49:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:49:36.124753 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf6dd71-f952-4d0b-9e6e-40b91c52edc0 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:36.124958 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4515c399-93a0-46f6-b9f5-bf2e837e77d3 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:36.124958 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4515c399-93a0-46f6-b9f5-bf2e837e77d3 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:36.124958 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4515c399-93a0-46f6-b9f5-bf2e837e77d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:36.156385 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-4515c399-93a0-46f6-b9f5-bf2e837e77d3 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:49:36.160217 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-4515c399-93a0-46f6-b9f5-bf2e837e77d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:49:36.263541 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 22/42] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:49:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 21:49:36.264863 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4515c399-93a0-46f6-b9f5-bf2e837e77d3 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:36.277221 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:36.278789 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf6dd71-f952-4d0b-9e6e-40b91c52edc0 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:36.317431 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13219df-6e28-4e78-b2fd-d2c8f5661ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:36.318072 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13219df-6e28-4e78-b2fd-d2c8f5661ad4 None admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:36.318365 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13219df-6e28-4e78-b2fd-d2c8f5661ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:36.318641 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13219df-6e28-4e78-b2fd-d2c8f5661ad4 None admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:36.319286 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13219df-6e28-4e78-b2fd-d2c8f5661ad4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:36.320823 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13219df-6e28-4e78-b2fd-d2c8f5661ad4 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:36.384384 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4515c399-93a0-46f6-b9f5-bf2e837e77d3 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:36.418396 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e9531e-a57d-46f8-9d39-5803900fd3d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:36.419011 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e9531e-a57d-46f8-9d39-5803900fd3d4 None admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:36.419296 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e9531e-a57d-46f8-9d39-5803900fd3d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:36.419736 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e9531e-a57d-46f8-9d39-5803900fd3d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:36.420474 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e9531e-a57d-46f8-9d39-5803900fd3d4 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:36.421947 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e9531e-a57d-46f8-9d39-5803900fd3d4 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:36.467786 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 21/43] 10.222.0.18 () {60 vars in 1141 bytes} [Sat Aug 6 21:49:36 2022] POST /identity/v3/projects => generated 301 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 21:49:36.468790 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13219df-6e28-4e78-b2fd-d2c8f5661ad4 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:36.513891 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:36.577939 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 23/44] 10.222.0.18 () {60 vars in 1131 bytes} [Sat Aug 6 21:49:36 2022] POST /identity/v3/roles => generated 223 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 21:49:36.605919 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13219df-6e28-4e78-b2fd-d2c8f5661ad4 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:36.648039 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e228fd-3494-42e0-b19a-56b7c05f6e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:36.649036 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e228fd-3494-42e0-b19a-56b7c05f6e55 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:36.649463 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e228fd-3494-42e0-b19a-56b7c05f6e55 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:36.649879 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e228fd-3494-42e0-b19a-56b7c05f6e55 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:36.651378 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e228fd-3494-42e0-b19a-56b7c05f6e55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:36.653192 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e228fd-3494-42e0-b19a-56b7c05f6e55 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:36.655040 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 22/45] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:49:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:49:36.668628 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e9531e-a57d-46f8-9d39-5803900fd3d4 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:36.753099 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:36.789509 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e9531e-a57d-46f8-9d39-5803900fd3d4 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:36.825513 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8760fe-66b8-4dd9-bc75-95a989436a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:36.827031 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8760fe-66b8-4dd9-bc75-95a989436a3b None admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:36.827531 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8760fe-66b8-4dd9-bc75-95a989436a3b None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:36.828072 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8760fe-66b8-4dd9-bc75-95a989436a3b None admin] PATH_INFO: `/v3/projects` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:36.828796 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8760fe-66b8-4dd9-bc75-95a989436a3b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:36.830264 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8760fe-66b8-4dd9-bc75-95a989436a3b None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:36.947949 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 24/46] 10.222.0.18 () {60 vars in 1141 bytes} [Sat Aug 6 21:49:36 2022] POST /identity/v3/projects => generated 312 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 21:49:38.198053 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555f604-2123-443f-ab14-a7ed8739171d None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:38.198602 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555f604-2123-443f-ab14-a7ed8739171d None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:38.198602 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555f604-2123-443f-ab14-a7ed8739171d None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:38.200089 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 23/47] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:49:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:49:38.205013 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13f74f6-af36-418e-b9e3-e95d0c65c6d7 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:38.205399 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13f74f6-af36-418e-b9e3-e95d0c65c6d7 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:38.205709 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13f74f6-af36-418e-b9e3-e95d0c65c6d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:38.224274 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-a13f74f6-af36-418e-b9e3-e95d0c65c6d7 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:49:38.225494 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-a13f74f6-af36-418e-b9e3-e95d0c65c6d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:49:38.289768 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 25/48] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:49:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 21:49:38.311525 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b555f604-2123-443f-ab14-a7ed8739171d None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:38.417384 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b555f604-2123-443f-ab14-a7ed8739171d None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:38.419035 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:38.451461 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df62502-d398-4551-b6f4-472bede9bb9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:38.452017 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df62502-d398-4551-b6f4-472bede9bb9b None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:38.452297 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df62502-d398-4551-b6f4-472bede9bb9b None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:38.452600 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df62502-d398-4551-b6f4-472bede9bb9b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:38.463587 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df62502-d398-4551-b6f4-472bede9bb9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:49:38.468328 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-9df62502-d398-4551-b6f4-472bede9bb9b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 21:49:38.468328 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:49:38.468328 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:49:38.468328 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:49:38.468328 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:49:38.468328 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:49:38.468328 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:49:38.468328 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:49:38.468328 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:49:38.468328 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:49:38.468328 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:49:38.468328 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:49:38.468328 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:49:38.468328 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:49:38.468328 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:49:38.468328 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:49:38.469179 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:49:38.469179 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:49:38.469179 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:49:38.469179 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:49:38.469179 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:49:38.469179 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:49:38.469179 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:49:38.469179 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:49:38.469179 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:49:38.469179 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:49:38.469179 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:49:38.469179 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:49:38.469179 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:49:38.469179 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:49:38.469179 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:49:38.469179 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:49:38.469916 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:49:38.469916 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:49:38.469916 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:49:38.469916 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:49:38.469916 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:49:38.469916 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:49:38.469916 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:49:38.469916 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:49:38.469916 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:49:38.469916 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:49:38.469916 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 21:49:38.469916 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:49:38.469916 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 24/49] 10.222.0.18 () {56 vars in 1096 bytes} [Sat Aug 6 21:49:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:49:38.475403 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13f74f6-af36-418e-b9e3-e95d0c65c6d7 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:38.524209 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:38.574474 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13f74f6-af36-418e-b9e3-e95d0c65c6d7 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:38.607814 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d85f023-fe58-4bf7-9e38-917e21534de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:38.608430 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d85f023-fe58-4bf7-9e38-917e21534de1 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:38.608720 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d85f023-fe58-4bf7-9e38-917e21534de1 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:38.608995 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d85f023-fe58-4bf7-9e38-917e21534de1 None admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:38.609660 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d85f023-fe58-4bf7-9e38-917e21534de1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:49:38.609960 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d85f023-fe58-4bf7-9e38-917e21534de1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:38.611333 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d85f023-fe58-4bf7-9e38-917e21534de1 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:38.620124 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 26/50] 10.222.0.18 () {56 vars in 1099 bytes} [Sat Aug 6 21:49:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:49:38.625429 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df62502-d398-4551-b6f4-472bede9bb9b None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:38.724429 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df62502-d398-4551-b6f4-472bede9bb9b None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:38.758155 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f31624-00a1-4749-9c4d-e177d305262d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:38.758784 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f31624-00a1-4749-9c4d-e177d305262d None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:38.759067 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f31624-00a1-4749-9c4d-e177d305262d None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:38.759344 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f31624-00a1-4749-9c4d-e177d305262d None admin] PATH_INFO: `/v3/users/6bb7ac25afcb4fcaba91aca891ca7fc9` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:38.760493 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f31624-00a1-4749-9c4d-e177d305262d None admin] RBAC: Authorizing `identity:get_user(user_id=6bb7ac25afcb4fcaba91aca891ca7fc9)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:38.761946 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f31624-00a1-4749-9c4d-e177d305262d None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:38.763501 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 25/51] 10.222.0.18 () {56 vars in 1177 bytes} [Sat Aug 6 21:49:38 2022] GET /identity/v3/users/6bb7ac25afcb4fcaba91aca891ca7fc9 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:49:38.768961 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d85f023-fe58-4bf7-9e38-917e21534de1 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:38.866901 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d85f023-fe58-4bf7-9e38-917e21534de1 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:38.879218 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:38.899456 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a48b66-1976-4dc2-9c15-53070125a07d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:38.900091 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a48b66-1976-4dc2-9c15-53070125a07d None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:38.900373 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a48b66-1976-4dc2-9c15-53070125a07d None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:38.900680 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a48b66-1976-4dc2-9c15-53070125a07d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:38.901793 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a48b66-1976-4dc2-9c15-53070125a07d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:38.903240 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a48b66-1976-4dc2-9c15-53070125a07d None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:38.904891 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 27/52] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:49:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:49:38.911069 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f31624-00a1-4749-9c4d-e177d305262d None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:38.941170 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:38.949812 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437fa8ee-f9cf-410d-bae1-747b4c02a2ec None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:38.950135 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437fa8ee-f9cf-410d-bae1-747b4c02a2ec None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:38.950415 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437fa8ee-f9cf-410d-bae1-747b4c02a2ec None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:38.951877 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 28/53] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:49:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:49:38.956555 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e990677-d48b-42c4-839a-232944b6cd4a None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:38.956885 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e990677-d48b-42c4-839a-232944b6cd4a None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:38.957224 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e990677-d48b-42c4-839a-232944b6cd4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:38.975212 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-1e990677-d48b-42c4-839a-232944b6cd4a None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:49:38.976322 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-1e990677-d48b-42c4-839a-232944b6cd4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:49:39.011191 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f31624-00a1-4749-9c4d-e177d305262d None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:39.042987 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 29/54] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:49:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 21:49:39.044474 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82404ea8-2b95-4b23-b0f4-52caf5721682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:39.045078 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82404ea8-2b95-4b23-b0f4-52caf5721682 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:39.045355 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82404ea8-2b95-4b23-b0f4-52caf5721682 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:39.045634 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82404ea8-2b95-4b23-b0f4-52caf5721682 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:39.046343 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82404ea8-2b95-4b23-b0f4-52caf5721682 None admin] RBAC: Adding query filter params (role.id=4b8dcf0491fd4072acc69fd894bf77d0, scope.domain.id=default, user.id=6bb7ac25afcb4fcaba91aca891ca7fc9) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:49:39.046645 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82404ea8-2b95-4b23-b0f4-52caf5721682 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:39.048072 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82404ea8-2b95-4b23-b0f4-52caf5721682 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:39.054619 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 26/55] 10.222.0.18 () {56 vars in 1322 bytes} [Sat Aug 6 21:49:38 2022] GET /identity/v3/role_assignments?user.id=6bb7ac25afcb4fcaba91aca891ca7fc9&scope.domain.id=default&role.id=4b8dcf0491fd4072acc69fd894bf77d0 => generated 530 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:49:39.066768 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e990677-d48b-42c4-839a-232944b6cd4a None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:39.174505 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e990677-d48b-42c4-839a-232944b6cd4a None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:39.186486 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:39.211502 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05638be-010e-408c-bd03-b67d6240b561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:39.212896 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05638be-010e-408c-bd03-b67d6240b561 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:39.212896 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05638be-010e-408c-bd03-b67d6240b561 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:39.212896 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05638be-010e-408c-bd03-b67d6240b561 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:39.213823 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05638be-010e-408c-bd03-b67d6240b561 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:39.215270 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05638be-010e-408c-bd03-b67d6240b561 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:39.216873 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 30/56] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:49:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:49:39.222630 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82404ea8-2b95-4b23-b0f4-52caf5721682 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:39.318372 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:39.324738 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82404ea8-2b95-4b23-b0f4-52caf5721682 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:39.356506 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44efa3c3-130f-4765-a9cc-e6ab34921bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:39.357164 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44efa3c3-130f-4765-a9cc-e6ab34921bf0 None admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:39.357699 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44efa3c3-130f-4765-a9cc-e6ab34921bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:39.357971 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44efa3c3-130f-4765-a9cc-e6ab34921bf0 None admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:39.358621 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44efa3c3-130f-4765-a9cc-e6ab34921bf0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:39.360068 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44efa3c3-130f-4765-a9cc-e6ab34921bf0 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:39.616719 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 27/57] 10.222.0.18 () {60 vars in 1140 bytes} [Sat Aug 6 21:49:39 2022] POST /identity/v3/projects => generated 298 bytes in 394 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 21:49:41.591514 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9982a4b6-4690-49f5-9297-4e16d314ae3a None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:41.592024 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9982a4b6-4690-49f5-9297-4e16d314ae3a None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:41.592108 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9982a4b6-4690-49f5-9297-4e16d314ae3a None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:41.593437 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 31/58] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:49:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:49:41.598509 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c521f20f-a802-4c14-9267-74801ef807f6 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:41.598924 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c521f20f-a802-4c14-9267-74801ef807f6 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:41.599067 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c521f20f-a802-4c14-9267-74801ef807f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:41.617874 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-c521f20f-a802-4c14-9267-74801ef807f6 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:49:41.619338 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-c521f20f-a802-4c14-9267-74801ef807f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:49:41.683928 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 28/59] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:49:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 21:49:41.708264 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9982a4b6-4690-49f5-9297-4e16d314ae3a None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:41.805804 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9982a4b6-4690-49f5-9297-4e16d314ae3a None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:41.838174 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76be3f1-cd6d-4f26-b975-e6004c798193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:41.838789 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76be3f1-cd6d-4f26-b975-e6004c798193 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:41.839070 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76be3f1-cd6d-4f26-b975-e6004c798193 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:41.839346 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76be3f1-cd6d-4f26-b975-e6004c798193 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:41.840472 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76be3f1-cd6d-4f26-b975-e6004c798193 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:41.841910 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76be3f1-cd6d-4f26-b975-e6004c798193 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:41.843433 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 32/60] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:49:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:49:41.849432 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c521f20f-a802-4c14-9267-74801ef807f6 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:41.922202 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:41.944390 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c521f20f-a802-4c14-9267-74801ef807f6 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:41.977611 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961f418f-0723-45fe-9ab3-a90f779c7128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:41.978207 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f418f-0723-45fe-9ab3-a90f779c7128 None admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:41.978488 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f418f-0723-45fe-9ab3-a90f779c7128 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:41.978761 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f418f-0723-45fe-9ab3-a90f779c7128 None admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:41.979402 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961f418f-0723-45fe-9ab3-a90f779c7128 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:41.980882 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961f418f-0723-45fe-9ab3-a90f779c7128 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:42.077036 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 29/61] 10.222.0.18 () {60 vars in 1132 bytes} [Sat Aug 6 21:49:41 2022] POST /identity/v3/users => generated 275 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 21:49:45.220130 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84c1318-1710-42d7-ad92-6f4ef7773b6f None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:45.220660 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84c1318-1710-42d7-ad92-6f4ef7773b6f None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:45.220733 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84c1318-1710-42d7-ad92-6f4ef7773b6f None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:45.222075 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 33/62] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:49:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:49:45.238758 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92239da7-01cc-4214-ada5-b828193f3033 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:45.239246 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92239da7-01cc-4214-ada5-b828193f3033 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:45.239315 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92239da7-01cc-4214-ada5-b828193f3033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:45.261326 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6163e0dc-3ed3-4ff6-92a0-0b5831fd9f07 None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:45.261943 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6163e0dc-3ed3-4ff6-92a0-0b5831fd9f07 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:45.262454 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6163e0dc-3ed3-4ff6-92a0-0b5831fd9f07 None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:45.263528 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-92239da7-01cc-4214-ada5-b828193f3033 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:49:45.264245 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 34/63] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:49:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:49:45.265437 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-92239da7-01cc-4214-ada5-b828193f3033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:49:45.287228 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aec2da-674e-4b7c-bba7-03b5bdf7c5e7 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:45.287859 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aec2da-674e-4b7c-bba7-03b5bdf7c5e7 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:45.288339 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aec2da-674e-4b7c-bba7-03b5bdf7c5e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:45.307388 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-25aec2da-674e-4b7c-bba7-03b5bdf7c5e7 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:49:45.308942 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-25aec2da-674e-4b7c-bba7-03b5bdf7c5e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:49:45.387458 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 30/64] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:49:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 21:49:45.396764 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:45.412256 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92239da7-01cc-4214-ada5-b828193f3033 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:45.444435 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 35/65] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:49:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 21:49:45.444435 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db3181e-7755-45e7-9af7-e693d9b29745 None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:45.444435 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db3181e-7755-45e7-9af7-e693d9b29745 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:45.444435 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db3181e-7755-45e7-9af7-e693d9b29745 None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:45.444435 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 36/66] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:49:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:49:45.446558 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb4d307-23c2-415d-bbca-3a2c9a7ef7c6 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:45.446953 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb4d307-23c2-415d-bbca-3a2c9a7ef7c6 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:45.447317 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb4d307-23c2-415d-bbca-3a2c9a7ef7c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:45.466631 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-4fb4d307-23c2-415d-bbca-3a2c9a7ef7c6 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:49:45.468108 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-4fb4d307-23c2-415d-bbca-3a2c9a7ef7c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:49:45.492944 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:45.537635 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92239da7-01cc-4214-ada5-b828193f3033 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:45.540969 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 37/67] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:49:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 21:49:45.544481 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb4d307-23c2-415d-bbca-3a2c9a7ef7c6 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:45.577770 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1e8313-7a5b-4d00-a26b-f21bb6d3e31a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:45.578913 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1e8313-7a5b-4d00-a26b-f21bb6d3e31a None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:45.579842 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1e8313-7a5b-4d00-a26b-f21bb6d3e31a None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:45.580265 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1e8313-7a5b-4d00-a26b-f21bb6d3e31a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:45.598750 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1e8313-7a5b-4d00-a26b-f21bb6d3e31a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:49:45.606041 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-ef1e8313-7a5b-4d00-a26b-f21bb6d3e31a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 21:49:45.606041 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:49:45.606041 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:49:45.606041 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:49:45.606041 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:49:45.606041 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:49:45.606041 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:49:45.606041 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:49:45.606041 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:49:45.606041 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:49:45.606041 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:49:45.606041 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:49:45.606041 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:49:45.606041 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:49:45.606041 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:49:45.606041 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:49:45.606897 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:49:45.606897 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:49:45.606897 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:49:45.606897 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:49:45.606897 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:49:45.606897 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:49:45.606897 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:49:45.606897 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:49:45.606897 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:49:45.606897 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:49:45.606897 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:49:45.606897 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:49:45.606897 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:49:45.606897 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:49:45.606897 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:49:45.606897 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:49:45.607908 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:49:45.607908 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:49:45.607908 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:49:45.607908 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:49:45.607908 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:49:45.607908 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:49:45.607908 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:49:45.607908 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:49:45.607908 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:49:45.607908 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:49:45.607908 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 21:49:45.607908 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:49:45.610608 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 31/68] 10.222.0.18 () {56 vars in 1096 bytes} [Sat Aug 6 21:49:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 21:49:45.611591 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1e8313-7a5b-4d00-a26b-f21bb6d3e31a None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:45.693322 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb4d307-23c2-415d-bbca-3a2c9a7ef7c6 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:45.702222 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:45.730868 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228e098b-38d3-45ad-a38b-8a64e73b8db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:45.731094 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1e8313-7a5b-4d00-a26b-f21bb6d3e31a None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:45.731526 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228e098b-38d3-45ad-a38b-8a64e73b8db3 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:45.731810 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228e098b-38d3-45ad-a38b-8a64e73b8db3 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:45.732086 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228e098b-38d3-45ad-a38b-8a64e73b8db3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:45.744269 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228e098b-38d3-45ad-a38b-8a64e73b8db3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:49:45.749573 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-228e098b-38d3-45ad-a38b-8a64e73b8db3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 21:49:45.749573 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:49:45.749573 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:49:45.749573 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:49:45.749573 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:49:45.749573 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:49:45.749573 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:49:45.749573 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:49:45.749573 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:49:45.749573 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:49:45.749573 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:49:45.749573 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:49:45.749573 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:49:45.749573 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:49:45.749573 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:49:45.749573 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:49:45.750401 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:49:45.750401 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:49:45.750401 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:49:45.750401 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:49:45.750401 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:49:45.750401 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:49:45.750401 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:49:45.750401 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:49:45.750401 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:49:45.750401 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:49:45.750401 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:49:45.750401 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:49:45.750401 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:49:45.750401 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:49:45.750401 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:49:45.750401 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:49:45.751247 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:49:45.751247 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:49:45.751247 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:49:45.751247 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:49:45.751247 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:49:45.751247 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:49:45.751247 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:49:45.751247 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:49:45.751247 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:49:45.751247 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:49:45.751247 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 21:49:45.751247 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:49:45.751247 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 38/69] 10.222.0.18 () {56 vars in 1099 bytes} [Sat Aug 6 21:49:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:49:45.751871 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228e098b-38d3-45ad-a38b-8a64e73b8db3 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:45.762101 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:45.774547 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb11045-4a39-41a2-ab89-3af380c76464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:45.775193 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb11045-4a39-41a2-ab89-3af380c76464 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:45.775481 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb11045-4a39-41a2-ab89-3af380c76464 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:45.775761 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb11045-4a39-41a2-ab89-3af380c76464 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:45.786444 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb11045-4a39-41a2-ab89-3af380c76464 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:49:45.793853 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-eeb11045-4a39-41a2-ab89-3af380c76464 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 21:49:45.793853 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:49:45.793853 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:49:45.793853 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:49:45.793853 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:49:45.793853 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:49:45.793853 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:49:45.793853 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:49:45.793853 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:49:45.793853 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:49:45.793853 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:49:45.793853 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:49:45.793853 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:49:45.793853 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:49:45.793853 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:49:45.793853 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:49:45.794671 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:49:45.794671 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:49:45.794671 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:49:45.794671 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:49:45.794671 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:49:45.794671 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:49:45.794671 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:49:45.794671 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:49:45.794671 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:49:45.794671 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:49:45.794671 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:49:45.794671 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:49:45.794671 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:49:45.794671 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:49:45.794671 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:49:45.794671 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:49:45.795429 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:49:45.795429 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:49:45.795429 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:49:45.795429 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:49:45.795429 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:49:45.795429 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:49:45.795429 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:49:45.795429 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:49:45.795429 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:49:45.795429 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:49:45.795429 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 21:49:45.795429 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:49:45.795429 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 32/70] 10.222.0.18 () {56 vars in 1114 bytes} [Sat Aug 6 21:49:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:49:45.795429 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c4d5b7-cc9a-46ce-ab93-dec4548fa76b None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:45.796112 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c4d5b7-cc9a-46ce-ab93-dec4548fa76b None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:45.796112 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c4d5b7-cc9a-46ce-ab93-dec4548fa76b None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:45.796112 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 33/71] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:49:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:49:45.797411 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c4d5b7-cc9a-46ce-ab93-dec4548fa76b None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:45.867106 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228e098b-38d3-45ad-a38b-8a64e73b8db3 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:45.899102 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c4d5b7-cc9a-46ce-ab93-dec4548fa76b None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:45.899608 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0546e341-78bd-4dee-a23e-ec9f50f97cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:45.900221 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0546e341-78bd-4dee-a23e-ec9f50f97cd6 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:45.900538 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0546e341-78bd-4dee-a23e-ec9f50f97cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:45.900817 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0546e341-78bd-4dee-a23e-ec9f50f97cd6 None admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:45.901486 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0546e341-78bd-4dee-a23e-ec9f50f97cd6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:49:45.901789 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0546e341-78bd-4dee-a23e-ec9f50f97cd6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:45.903178 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0546e341-78bd-4dee-a23e-ec9f50f97cd6 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:45.911611 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 39/72] 10.222.0.18 () {56 vars in 1099 bytes} [Sat Aug 6 21:49:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:49:45.912427 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0546e341-78bd-4dee-a23e-ec9f50f97cd6 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:45.937649 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766df103-2e92-4f4b-9fab-33786518b57e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:45.938257 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766df103-2e92-4f4b-9fab-33786518b57e None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:45.938544 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766df103-2e92-4f4b-9fab-33786518b57e None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:45.938821 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766df103-2e92-4f4b-9fab-33786518b57e None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:45.939511 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766df103-2e92-4f4b-9fab-33786518b57e None admin] RBAC: Adding query filter params (name=member) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:49:45.939845 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766df103-2e92-4f4b-9fab-33786518b57e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:45.940457 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:45.941810 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766df103-2e92-4f4b-9fab-33786518b57e None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:45.950355 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 34/73] 10.222.0.18 () {56 vars in 1101 bytes} [Sat Aug 6 21:49:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:49:45.950355 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630d4544-5dbc-4f27-9bd6-560c9ee9c15c None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:45.950355 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630d4544-5dbc-4f27-9bd6-560c9ee9c15c None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:45.950355 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630d4544-5dbc-4f27-9bd6-560c9ee9c15c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:45.970637 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-630d4544-5dbc-4f27-9bd6-560c9ee9c15c None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:49:45.972583 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-630d4544-5dbc-4f27-9bd6-560c9ee9c15c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:49:46.013576 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:46.018105 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0546e341-78bd-4dee-a23e-ec9f50f97cd6 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:46.039218 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 35/74] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:49:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 21:49:46.040611 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22260598-021e-4bbc-8c37-11578aefc1c3 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:46.040917 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22260598-021e-4bbc-8c37-11578aefc1c3 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:46.041173 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22260598-021e-4bbc-8c37-11578aefc1c3 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:46.044569 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 36/75] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:49:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:49:46.044569 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22260598-021e-4bbc-8c37-11578aefc1c3 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:46.055568 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf28869-6607-4046-a724-671d79d84310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:46.056167 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf28869-6607-4046-a724-671d79d84310 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:46.056462 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf28869-6607-4046-a724-671d79d84310 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:46.056737 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf28869-6607-4046-a724-671d79d84310 None admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:46.057402 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf28869-6607-4046-a724-671d79d84310 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:49:46.057702 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf28869-6607-4046-a724-671d79d84310 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:46.059086 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf28869-6607-4046-a724-671d79d84310 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:46.067865 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 40/76] 10.222.0.18 () {56 vars in 1111 bytes} [Sat Aug 6 21:49:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:49:46.069031 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf28869-6607-4046-a724-671d79d84310 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:46.157647 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:46.161276 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22260598-021e-4bbc-8c37-11578aefc1c3 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:46.167183 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf28869-6607-4046-a724-671d79d84310 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:46.194405 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4030dcd7-0719-4e6b-9c71-42be8d56bbb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:46.195123 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4030dcd7-0719-4e6b-9c71-42be8d56bbb4 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:46.195476 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4030dcd7-0719-4e6b-9c71-42be8d56bbb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:46.195822 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4030dcd7-0719-4e6b-9c71-42be8d56bbb4 None admin] PATH_INFO: `/v3/users/6bb7ac25afcb4fcaba91aca891ca7fc9` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:46.197096 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4030dcd7-0719-4e6b-9c71-42be8d56bbb4 None admin] RBAC: Authorizing `identity:get_user(user_id=6bb7ac25afcb4fcaba91aca891ca7fc9)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:46.198595 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4030dcd7-0719-4e6b-9c71-42be8d56bbb4 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:46.199580 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4793263e-fc7e-43cc-a526-f97611241474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:46.200168 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 37/77] 10.222.0.18 () {56 vars in 1177 bytes} [Sat Aug 6 21:49:46 2022] GET /identity/v3/users/6bb7ac25afcb4fcaba91aca891ca7fc9 => generated 247 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:49:46.200288 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4793263e-fc7e-43cc-a526-f97611241474 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:46.200506 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4793263e-fc7e-43cc-a526-f97611241474 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:46.200782 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4793263e-fc7e-43cc-a526-f97611241474 None admin] PATH_INFO: `/v3/users/b7b3753a7de94d39b55d2106b904b48d` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:46.201315 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd86aea-7b1d-4c04-b3f0-57f436417056 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:46.201593 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd86aea-7b1d-4c04-b3f0-57f436417056 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:46.201868 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd86aea-7b1d-4c04-b3f0-57f436417056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:46.216833 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4793263e-fc7e-43cc-a526-f97611241474 None admin] RBAC: Authorizing `identity:get_user(user_id=b7b3753a7de94d39b55d2106b904b48d)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:46.218295 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4793263e-fc7e-43cc-a526-f97611241474 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:46.219936 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 41/78] 10.222.0.18 () {56 vars in 1177 bytes} [Sat Aug 6 21:49:46 2022] GET /identity/v3/users/b7b3753a7de94d39b55d2106b904b48d => generated 275 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:49:46.225270 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-7dd86aea-7b1d-4c04-b3f0-57f436417056 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:49:46.225364 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4793263e-fc7e-43cc-a526-f97611241474 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:46.225447 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-7dd86aea-7b1d-4c04-b3f0-57f436417056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:49:46.256653 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:46.287857 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 38/79] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:49:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 21:49:46.288656 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd86aea-7b1d-4c04-b3f0-57f436417056 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:46.323192 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4793263e-fc7e-43cc-a526-f97611241474 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:46.358399 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb82441-f859-4bc7-b1a6-8ad2558edcbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:46.359041 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb82441-f859-4bc7-b1a6-8ad2558edcbe None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:46.359312 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb82441-f859-4bc7-b1a6-8ad2558edcbe None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:46.359583 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb82441-f859-4bc7-b1a6-8ad2558edcbe None admin] PATH_INFO: `/v3/roles/member` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:46.369759 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb82441-f859-4bc7-b1a6-8ad2558edcbe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:49:46.374451 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-4cb82441-f859-4bc7-b1a6-8ad2558edcbe None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 21:49:46.374451 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:49:46.374451 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:49:46.374451 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:49:46.374451 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:49:46.374451 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:49:46.374451 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:49:46.374451 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:49:46.374451 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:49:46.374451 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:49:46.374451 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:49:46.374451 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:49:46.374451 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:49:46.374451 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:49:46.374451 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:49:46.374451 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:49:46.375273 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:49:46.375273 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:49:46.375273 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:49:46.375273 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:49:46.375273 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:49:46.375273 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:49:46.375273 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:49:46.375273 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:49:46.375273 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:49:46.375273 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:49:46.375273 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:49:46.375273 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:49:46.375273 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:49:46.375273 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:49:46.375273 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:49:46.375273 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:49:46.381768 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:49:46.381768 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:49:46.381768 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:49:46.381768 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:49:46.381768 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:49:46.381768 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:49:46.381768 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:49:46.381768 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:49:46.381768 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:49:46.381768 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:49:46.381768 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 21:49:46.381768 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:49:46.381768 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 42/80] 10.222.0.18 () {56 vars in 1099 bytes} [Sat Aug 6 21:49:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:49:46.381768 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb82441-f859-4bc7-b1a6-8ad2558edcbe None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:46.388765 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:46.395840 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd86aea-7b1d-4c04-b3f0-57f436417056 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:46.429224 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4dbb18-271e-4155-960b-85a6e8c6ab0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:46.429855 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4dbb18-271e-4155-960b-85a6e8c6ab0d None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:46.430136 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4dbb18-271e-4155-960b-85a6e8c6ab0d None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:46.430408 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4dbb18-271e-4155-960b-85a6e8c6ab0d None admin] PATH_INFO: `/v3/users/b7b3753a7de94d39b55d2106b904b48d` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:46.431515 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4dbb18-271e-4155-960b-85a6e8c6ab0d None admin] RBAC: Authorizing `identity:get_user(user_id=b7b3753a7de94d39b55d2106b904b48d)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:46.432994 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4dbb18-271e-4155-960b-85a6e8c6ab0d None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:46.434554 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 39/81] 10.222.0.18 () {56 vars in 1177 bytes} [Sat Aug 6 21:49:46 2022] GET /identity/v3/users/b7b3753a7de94d39b55d2106b904b48d => generated 275 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:49:46.435305 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4dbb18-271e-4155-960b-85a6e8c6ab0d None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:46.476774 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:46.477092 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb82441-f859-4bc7-b1a6-8ad2558edcbe None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:46.512268 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e8cd91-f585-4c96-b834-b76e9151462f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:46.512914 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e8cd91-f585-4c96-b834-b76e9151462f None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:46.513201 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e8cd91-f585-4c96-b834-b76e9151462f None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:46.513479 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e8cd91-f585-4c96-b834-b76e9151462f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:46.534627 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4dbb18-271e-4155-960b-85a6e8c6ab0d None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:46.569092 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5498ac7-e0d7-4af0-8237-1db5c36e2738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:46.569628 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5498ac7-e0d7-4af0-8237-1db5c36e2738 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:46.569903 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5498ac7-e0d7-4af0-8237-1db5c36e2738 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:46.570174 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5498ac7-e0d7-4af0-8237-1db5c36e2738 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:46.581059 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e8cd91-f585-4c96-b834-b76e9151462f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:46.582795 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e8cd91-f585-4c96-b834-b76e9151462f None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:46.592126 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:46.639816 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5498ac7-e0d7-4af0-8237-1db5c36e2738 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:46.641339 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5498ac7-e0d7-4af0-8237-1db5c36e2738 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:46.680755 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 43/82] 10.222.0.18 () {58 vars in 1282 bytes} [Sat Aug 6 21:49:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 21:49:46.681595 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e8cd91-f585-4c96-b834-b76e9151462f None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:46.686311 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:46.745854 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 40/83] 10.222.0.18 () {58 vars in 1282 bytes} [Sat Aug 6 21:49:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 21:49:46.745854 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5498ac7-e0d7-4af0-8237-1db5c36e2738 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:46.781510 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e8cd91-f585-4c96-b834-b76e9151462f None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:46.790486 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:46.814929 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d1b8b6-ba41-4343-a49d-560fce38010c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:46.815554 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d1b8b6-ba41-4343-a49d-560fce38010c None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:46.815829 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d1b8b6-ba41-4343-a49d-560fce38010c None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:46.816670 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d1b8b6-ba41-4343-a49d-560fce38010c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:46.817783 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d1b8b6-ba41-4343-a49d-560fce38010c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:46.819296 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d1b8b6-ba41-4343-a49d-560fce38010c None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:46.821595 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 44/84] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:49:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:49:46.822748 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d1b8b6-ba41-4343-a49d-560fce38010c None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:46.844545 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5498ac7-e0d7-4af0-8237-1db5c36e2738 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:46.880850 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec805fd7-2ce7-4616-bfb9-292fb7f09f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:46.881451 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec805fd7-2ce7-4616-bfb9-292fb7f09f89 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:46.881793 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec805fd7-2ce7-4616-bfb9-292fb7f09f89 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:46.882071 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec805fd7-2ce7-4616-bfb9-292fb7f09f89 None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:46.882732 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec805fd7-2ce7-4616-bfb9-292fb7f09f89 None admin] RBAC: Adding query filter params (name=member) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:49:46.883035 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec805fd7-2ce7-4616-bfb9-292fb7f09f89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:46.884447 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec805fd7-2ce7-4616-bfb9-292fb7f09f89 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:46.892739 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 41/85] 10.222.0.18 () {56 vars in 1101 bytes} [Sat Aug 6 21:49:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:49:46.893521 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec805fd7-2ce7-4616-bfb9-292fb7f09f89 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:46.896071 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:46.922856 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d1b8b6-ba41-4343-a49d-560fce38010c None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:46.957456 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5836390d-935d-43cb-aa44-bae5729d8bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:46.958062 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5836390d-935d-43cb-aa44-bae5729d8bfc None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:46.958346 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5836390d-935d-43cb-aa44-bae5729d8bfc None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:46.958619 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5836390d-935d-43cb-aa44-bae5729d8bfc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:46.993509 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec805fd7-2ce7-4616-bfb9-292fb7f09f89 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:47.002815 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:47.025762 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5836390d-935d-43cb-aa44-bae5729d8bfc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:47.026967 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76de922-f711-4ef8-85f7-6eb875588f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:47.027236 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5836390d-935d-43cb-aa44-bae5729d8bfc None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:47.027598 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76de922-f711-4ef8-85f7-6eb875588f87 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:47.027881 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76de922-f711-4ef8-85f7-6eb875588f87 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:47.028157 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76de922-f711-4ef8-85f7-6eb875588f87 None admin] PATH_INFO: `/v3/projects/edee8005e8714a9db1c10cfec12857b6` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:47.029312 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76de922-f711-4ef8-85f7-6eb875588f87 None admin] RBAC: Authorizing `identity:get_project(project_id=edee8005e8714a9db1c10cfec12857b6)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:47.030806 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76de922-f711-4ef8-85f7-6eb875588f87 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:47.032458 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 42/86] 10.222.0.18 () {56 vars in 1186 bytes} [Sat Aug 6 21:49:46 2022] GET /identity/v3/projects/edee8005e8714a9db1c10cfec12857b6 => generated 298 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:49:47.036784 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76de922-f711-4ef8-85f7-6eb875588f87 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:47.106968 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:47.123739 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 45/87] 10.222.0.18 () {58 vars in 1282 bytes} [Sat Aug 6 21:49:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 21:49:47.125260 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5836390d-935d-43cb-aa44-bae5729d8bfc None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:47.138055 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76de922-f711-4ef8-85f7-6eb875588f87 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:47.171519 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b51a46c-adad-400c-b6ea-0a1c8265b93d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:47.172136 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b51a46c-adad-400c-b6ea-0a1c8265b93d None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:47.172440 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b51a46c-adad-400c-b6ea-0a1c8265b93d None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:47.172715 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b51a46c-adad-400c-b6ea-0a1c8265b93d None admin] PATH_INFO: `/v3/projects/edee8005e8714a9db1c10cfec12857b6` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:47.173808 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b51a46c-adad-400c-b6ea-0a1c8265b93d None admin] RBAC: Authorizing `identity:get_project(project_id=edee8005e8714a9db1c10cfec12857b6)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:47.175237 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b51a46c-adad-400c-b6ea-0a1c8265b93d None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:47.176983 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 43/88] 10.222.0.18 () {56 vars in 1186 bytes} [Sat Aug 6 21:49:47 2022] GET /identity/v3/projects/edee8005e8714a9db1c10cfec12857b6 => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:49:47.177740 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b51a46c-adad-400c-b6ea-0a1c8265b93d None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:47.204187 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:47.226884 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5836390d-935d-43cb-aa44-bae5729d8bfc None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:47.260616 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3d7453-20c8-4ebd-b106-cf6f18e99700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:47.261216 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d7453-20c8-4ebd-b106-cf6f18e99700 None admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:47.261493 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d7453-20c8-4ebd-b106-cf6f18e99700 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:47.261767 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d7453-20c8-4ebd-b106-cf6f18e99700 None admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:47.262405 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3d7453-20c8-4ebd-b106-cf6f18e99700 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:47.263830 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3d7453-20c8-4ebd-b106-cf6f18e99700 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:47.279282 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b51a46c-adad-400c-b6ea-0a1c8265b93d None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:47.312732 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c7445b-db96-42ba-a7e7-e41af2832aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:47.313358 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c7445b-db96-42ba-a7e7-e41af2832aab None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:47.313641 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c7445b-db96-42ba-a7e7-e41af2832aab None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:47.313918 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c7445b-db96-42ba-a7e7-e41af2832aab None admin] PATH_INFO: `/v3/users/b7b3753a7de94d39b55d2106b904b48d` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:47.315042 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c7445b-db96-42ba-a7e7-e41af2832aab None admin] RBAC: Authorizing `identity:get_user(user_id=b7b3753a7de94d39b55d2106b904b48d)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:47.316572 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c7445b-db96-42ba-a7e7-e41af2832aab None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:47.318141 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 44/89] 10.222.0.18 () {56 vars in 1177 bytes} [Sat Aug 6 21:49:47 2022] GET /identity/v3/users/b7b3753a7de94d39b55d2106b904b48d => generated 275 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:49:47.318915 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c7445b-db96-42ba-a7e7-e41af2832aab None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:47.327413 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:47.359833 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 46/90] 10.222.0.18 () {60 vars in 1132 bytes} [Sat Aug 6 21:49:47 2022] POST /identity/v3/users => generated 289 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 21:49:47.380265 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3d7453-20c8-4ebd-b106-cf6f18e99700 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:47.425110 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c7445b-db96-42ba-a7e7-e41af2832aab None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:47.464238 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588fd04f-cd85-45e9-956c-b38b2b32ed47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:47.464890 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588fd04f-cd85-45e9-956c-b38b2b32ed47 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:47.465177 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588fd04f-cd85-45e9-956c-b38b2b32ed47 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:47.465454 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588fd04f-cd85-45e9-956c-b38b2b32ed47 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:47.466199 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588fd04f-cd85-45e9-956c-b38b2b32ed47 None admin] RBAC: Adding query filter params (role.id=4b8dcf0491fd4072acc69fd894bf77d0, scope.project.id=edee8005e8714a9db1c10cfec12857b6, user.id=6bb7ac25afcb4fcaba91aca891ca7fc9) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:49:47.466504 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588fd04f-cd85-45e9-956c-b38b2b32ed47 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:47.467959 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588fd04f-cd85-45e9-956c-b38b2b32ed47 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:47.484433 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 45/91] 10.222.0.18 () {56 vars in 1374 bytes} [Sat Aug 6 21:49:47 2022] GET /identity/v3/role_assignments?user.id=6bb7ac25afcb4fcaba91aca891ca7fc9&scope.project.id=edee8005e8714a9db1c10cfec12857b6&role.id=4b8dcf0491fd4072acc69fd894bf77d0 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:49:47.484433 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588fd04f-cd85-45e9-956c-b38b2b32ed47 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:47.516712 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3d7453-20c8-4ebd-b106-cf6f18e99700 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:47.562220 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:47.570369 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42d5a79-1b72-48f0-9b01-66151b955fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:47.570592 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42d5a79-1b72-48f0-9b01-66151b955fee None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:47.570819 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42d5a79-1b72-48f0-9b01-66151b955fee None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:47.571108 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42d5a79-1b72-48f0-9b01-66151b955fee None admin] PATH_INFO: `/v3/projects/edee8005e8714a9db1c10cfec12857b6` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:47.572280 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42d5a79-1b72-48f0-9b01-66151b955fee None admin] RBAC: Authorizing `identity:get_project(project_id=edee8005e8714a9db1c10cfec12857b6)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:47.573935 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42d5a79-1b72-48f0-9b01-66151b955fee None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:47.575699 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 47/92] 10.222.0.18 () {56 vars in 1186 bytes} [Sat Aug 6 21:49:47 2022] GET /identity/v3/projects/edee8005e8714a9db1c10cfec12857b6 => generated 298 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:49:47.577431 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42d5a79-1b72-48f0-9b01-66151b955fee None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:47.584537 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:47.625886 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588fd04f-cd85-45e9-956c-b38b2b32ed47 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:47.678202 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4978bea9-73b2-41d8-bb67-fbd99ae2a131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:47.678826 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4978bea9-73b2-41d8-bb67-fbd99ae2a131 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:47.679107 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4978bea9-73b2-41d8-bb67-fbd99ae2a131 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:47.679388 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4978bea9-73b2-41d8-bb67-fbd99ae2a131 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:47.680124 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4978bea9-73b2-41d8-bb67-fbd99ae2a131 None admin] RBAC: Adding query filter params (role.id=56e4a96094e9434cbbafb22579c034f5, scope.project.id=edee8005e8714a9db1c10cfec12857b6, user.id=b7b3753a7de94d39b55d2106b904b48d) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:49:47.680586 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4978bea9-73b2-41d8-bb67-fbd99ae2a131 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:47.682061 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4978bea9-73b2-41d8-bb67-fbd99ae2a131 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:47.689212 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 46/93] 10.222.0.18 () {56 vars in 1374 bytes} [Sat Aug 6 21:49:47 2022] GET /identity/v3/role_assignments?user.id=b7b3753a7de94d39b55d2106b904b48d&scope.project.id=edee8005e8714a9db1c10cfec12857b6&role.id=56e4a96094e9434cbbafb22579c034f5 => generated 259 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:49:47.689769 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4978bea9-73b2-41d8-bb67-fbd99ae2a131 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:47.722751 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42d5a79-1b72-48f0-9b01-66151b955fee None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:47.771198 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b50219-d37e-4fd5-82c4-ca4f61ded455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:47.771877 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b50219-d37e-4fd5-82c4-ca4f61ded455 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:47.772162 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b50219-d37e-4fd5-82c4-ca4f61ded455 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:47.774217 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b50219-d37e-4fd5-82c4-ca4f61ded455 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:47.834963 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4978bea9-73b2-41d8-bb67-fbd99ae2a131 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:47.857818 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:47.864789 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b50219-d37e-4fd5-82c4-ca4f61ded455 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:47.864789 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b50219-d37e-4fd5-82c4-ca4f61ded455 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:47.891416 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e595fa-48fb-4ebb-be8f-c4b22fc23111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:47.892524 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e595fa-48fb-4ebb-be8f-c4b22fc23111 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:47.892914 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e595fa-48fb-4ebb-be8f-c4b22fc23111 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:47.893303 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e595fa-48fb-4ebb-be8f-c4b22fc23111 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:47.894240 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e595fa-48fb-4ebb-be8f-c4b22fc23111 None admin] RBAC: Adding query filter params (role.id=93c058ac49e242fd8517c80c552dea2d, scope.project.id=edee8005e8714a9db1c10cfec12857b6, user.id=b7b3753a7de94d39b55d2106b904b48d) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:49:47.895348 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e595fa-48fb-4ebb-be8f-c4b22fc23111 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:47.897713 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e595fa-48fb-4ebb-be8f-c4b22fc23111 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:47.899915 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:47.906635 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 47/94] 10.222.0.18 () {56 vars in 1374 bytes} [Sat Aug 6 21:49:47 2022] GET /identity/v3/role_assignments?user.id=b7b3753a7de94d39b55d2106b904b48d&scope.project.id=edee8005e8714a9db1c10cfec12857b6&role.id=93c058ac49e242fd8517c80c552dea2d => generated 259 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:49:47.998252 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 48/95] 10.222.0.18 () {58 vars in 1282 bytes} [Sat Aug 6 21:49:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 21:49:48.004272 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e595fa-48fb-4ebb-be8f-c4b22fc23111 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:48.155299 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e595fa-48fb-4ebb-be8f-c4b22fc23111 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:48.206233 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8439baa7-bdde-444c-9f9e-c2c91a5b5450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:48.207449 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8439baa7-bdde-444c-9f9e-c2c91a5b5450 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:48.207734 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8439baa7-bdde-444c-9f9e-c2c91a5b5450 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:48.208064 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8439baa7-bdde-444c-9f9e-c2c91a5b5450 None admin] PATH_INFO: `/v3/projects/f89b12d09e6449f990095cb05e6b5627` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:48.209299 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8439baa7-bdde-444c-9f9e-c2c91a5b5450 None admin] RBAC: Authorizing `identity:get_project(project_id=f89b12d09e6449f990095cb05e6b5627)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:48.210768 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8439baa7-bdde-444c-9f9e-c2c91a5b5450 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:48.212438 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 48/96] 10.222.0.18 () {56 vars in 1186 bytes} [Sat Aug 6 21:49:48 2022] GET /identity/v3/projects/f89b12d09e6449f990095cb05e6b5627 => generated 312 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:49:48.218433 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b50219-d37e-4fd5-82c4-ca4f61ded455 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:48.326016 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:48.329210 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b50219-d37e-4fd5-82c4-ca4f61ded455 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:48.376056 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f353131f-4401-47f0-87c0-1eda8b6c65f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:48.376702 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f353131f-4401-47f0-87c0-1eda8b6c65f8 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:48.376986 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f353131f-4401-47f0-87c0-1eda8b6c65f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:48.377312 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f353131f-4401-47f0-87c0-1eda8b6c65f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:48.378047 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f353131f-4401-47f0-87c0-1eda8b6c65f8 None admin] RBAC: Adding query filter params (role.id=56e4a96094e9434cbbafb22579c034f5, scope.project.id=f89b12d09e6449f990095cb05e6b5627, user.id=b7b3753a7de94d39b55d2106b904b48d) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:49:48.378408 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f353131f-4401-47f0-87c0-1eda8b6c65f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:48.380625 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f353131f-4401-47f0-87c0-1eda8b6c65f8 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:48.389725 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 49/97] 10.222.0.18 () {56 vars in 1374 bytes} [Sat Aug 6 21:49:48 2022] GET /identity/v3/role_assignments?user.id=b7b3753a7de94d39b55d2106b904b48d&scope.project.id=f89b12d09e6449f990095cb05e6b5627&role.id=56e4a96094e9434cbbafb22579c034f5 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:49:50.258398 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daffe4c-faf8-4aac-af15-353ab1154fa9 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:50.258944 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daffe4c-faf8-4aac-af15-353ab1154fa9 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:50.258944 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daffe4c-faf8-4aac-af15-353ab1154fa9 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:50.260523 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 49/98] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:49:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:49:50.266101 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d2f736-d617-4813-9deb-6f0eeb727df1 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:50.266501 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d2f736-d617-4813-9deb-6f0eeb727df1 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:50.266698 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d2f736-d617-4813-9deb-6f0eeb727df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:50.296247 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa24573-e32e-47ed-8ff0-1a727d82dd7f None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:50.296894 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa24573-e32e-47ed-8ff0-1a727d82dd7f None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:50.297360 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa24573-e32e-47ed-8ff0-1a727d82dd7f None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:50.297647 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-c3d2f736-d617-4813-9deb-6f0eeb727df1 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:49:50.298851 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-c3d2f736-d617-4813-9deb-6f0eeb727df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:49:50.299338 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 50/99] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:49:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:49:50.304845 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cc24cb-9baf-4aa7-8b3e-0b7518e22e58 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:50.305383 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cc24cb-9baf-4aa7-8b3e-0b7518e22e58 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:50.305870 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cc24cb-9baf-4aa7-8b3e-0b7518e22e58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:50.324528 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-18cc24cb-9baf-4aa7-8b3e-0b7518e22e58 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:49:50.326010 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-18cc24cb-9baf-4aa7-8b3e-0b7518e22e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:49:50.415927 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:50.431085 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 51/100] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:49:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 21:49:50.436692 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3849f3-da29-4b50-878f-43de35c1c10c None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:50.438652 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 50/101] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:49:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 21:49:50.439191 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3849f3-da29-4b50-878f-43de35c1c10c None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:50.444424 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3849f3-da29-4b50-878f-43de35c1c10c None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:50.445961 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 52/102] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:49:50 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:49:50.451339 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5166b60-1be6-4b47-af6d-0dc0b9c49ba5 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:50.451879 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5166b60-1be6-4b47-af6d-0dc0b9c49ba5 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:50.452362 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5166b60-1be6-4b47-af6d-0dc0b9c49ba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:50.471637 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3849f3-da29-4b50-878f-43de35c1c10c None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:50.477499 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-f5166b60-1be6-4b47-af6d-0dc0b9c49ba5 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:49:50.478732 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-f5166b60-1be6-4b47-af6d-0dc0b9c49ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:49:50.584731 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3849f3-da29-4b50-878f-43de35c1c10c None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:50.605762 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 51/103] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:49:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 21:49:50.607616 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5166b60-1be6-4b47-af6d-0dc0b9c49ba5 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:50.649836 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c442753-13b7-4e95-91ca-d3a0c7faca2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:50.650984 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c442753-13b7-4e95-91ca-d3a0c7faca2c None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:50.651790 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c442753-13b7-4e95-91ca-d3a0c7faca2c None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:50.652286 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c442753-13b7-4e95-91ca-d3a0c7faca2c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:50.665103 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:50.674902 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c442753-13b7-4e95-91ca-d3a0c7faca2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:49:50.684588 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-8c442753-13b7-4e95-91ca-d3a0c7faca2c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 21:49:50.684588 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:49:50.684588 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:49:50.684588 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:49:50.684588 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:49:50.684588 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:49:50.684588 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:49:50.684588 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:49:50.684588 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:49:50.684588 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:49:50.684588 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:49:50.684588 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:49:50.684588 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:49:50.684588 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:49:50.684588 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:49:50.684588 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:49:50.685460 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:49:50.685460 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:49:50.685460 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:49:50.685460 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:49:50.685460 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:49:50.685460 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:49:50.685460 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:49:50.685460 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:49:50.685460 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:49:50.685460 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:49:50.685460 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:49:50.685460 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:49:50.685460 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:49:50.685460 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:49:50.685460 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:49:50.685460 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:49:50.686226 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:49:50.686226 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:49:50.686226 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:49:50.686226 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:49:50.686226 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:49:50.686226 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:49:50.686226 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:49:50.686226 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:49:50.686226 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:49:50.686226 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:49:50.686226 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 21:49:50.686226 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:49:50.686226 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 53/104] 10.222.0.18 () {56 vars in 1099 bytes} [Sat Aug 6 21:49:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 21:49:50.686857 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c442753-13b7-4e95-91ca-d3a0c7faca2c None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:50.810284 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:50.813674 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5166b60-1be6-4b47-af6d-0dc0b9c49ba5 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:50.875745 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c442753-13b7-4e95-91ca-d3a0c7faca2c None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:50.895420 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52380641-7e57-4ec2-b94a-1a6e3f75a534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:50.896070 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52380641-7e57-4ec2-b94a-1a6e3f75a534 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:50.896349 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52380641-7e57-4ec2-b94a-1a6e3f75a534 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:50.896646 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52380641-7e57-4ec2-b94a-1a6e3f75a534 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:50.897765 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52380641-7e57-4ec2-b94a-1a6e3f75a534 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:50.899229 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52380641-7e57-4ec2-b94a-1a6e3f75a534 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:50.906475 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 52/105] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:49:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:49:50.907345 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52380641-7e57-4ec2-b94a-1a6e3f75a534 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:50.948831 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcaa5b7-f182-40c7-8b3d-499683547d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:50.949465 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcaa5b7-f182-40c7-8b3d-499683547d56 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:50.949739 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcaa5b7-f182-40c7-8b3d-499683547d56 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:50.950008 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcaa5b7-f182-40c7-8b3d-499683547d56 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:50.966617 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcaa5b7-f182-40c7-8b3d-499683547d56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:49:50.971265 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-bdcaa5b7-f182-40c7-8b3d-499683547d56 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 21:49:50.971265 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:49:50.971265 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:49:50.971265 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:49:50.971265 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:49:50.971265 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:49:50.971265 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:49:50.971265 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:49:50.971265 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:49:50.971265 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:49:50.971265 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:49:50.971265 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:49:50.971265 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:49:50.971265 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:49:50.971265 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:49:50.971265 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:49:50.972758 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:49:50.972758 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:49:50.972758 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:49:50.972758 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:49:50.972758 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:49:50.972758 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:49:50.972758 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:49:50.972758 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:49:50.972758 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:49:50.972758 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:49:50.972758 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:49:50.972758 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:49:50.972758 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:49:50.972758 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:49:50.972758 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:49:50.972758 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:49:50.973515 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:49:50.973515 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:49:50.973515 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:49:50.973515 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:49:50.973515 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:49:50.973515 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:49:50.973515 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:49:50.973515 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:49:50.973515 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:49:50.973515 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:49:50.973515 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 21:49:50.973515 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:49:50.974356 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 54/106] 10.222.0.18 () {56 vars in 1096 bytes} [Sat Aug 6 21:49:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:49:50.976003 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcaa5b7-f182-40c7-8b3d-499683547d56 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:51.087429 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:51.102720 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52380641-7e57-4ec2-b94a-1a6e3f75a534 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:51.158242 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1557928-a6e4-4891-b506-b8278d37a91e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:51.159198 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1557928-a6e4-4891-b506-b8278d37a91e None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:51.160674 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1557928-a6e4-4891-b506-b8278d37a91e None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:51.161093 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1557928-a6e4-4891-b506-b8278d37a91e None admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:51.162246 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1557928-a6e4-4891-b506-b8278d37a91e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:49:51.162893 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1557928-a6e4-4891-b506-b8278d37a91e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:51.165010 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1557928-a6e4-4891-b506-b8278d37a91e None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:51.166779 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcaa5b7-f182-40c7-8b3d-499683547d56 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:51.179464 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 53/107] 10.222.0.18 () {56 vars in 1101 bytes} [Sat Aug 6 21:49:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:49:51.179464 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1557928-a6e4-4891-b506-b8278d37a91e None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:51.223321 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:51.242377 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c43f91-6912-4638-b52a-3a2ce0954529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:51.243052 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c43f91-6912-4638-b52a-3a2ce0954529 None admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:51.243336 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c43f91-6912-4638-b52a-3a2ce0954529 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:51.243607 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c43f91-6912-4638-b52a-3a2ce0954529 None admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:51.244303 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c43f91-6912-4638-b52a-3a2ce0954529 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:51.245815 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c43f91-6912-4638-b52a-3a2ce0954529 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:51.295098 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1557928-a6e4-4891-b506-b8278d37a91e None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:51.328594 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bcb3e6-6a6a-43ef-8af8-ae7187c6bdf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:51.329140 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 55/108] 10.222.0.18 () {60 vars in 1141 bytes} [Sat Aug 6 21:49:50 2022] POST /identity/v3/projects => generated 302 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 21:49:51.329539 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bcb3e6-6a6a-43ef-8af8-ae7187c6bdf6 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:51.329539 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bcb3e6-6a6a-43ef-8af8-ae7187c6bdf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:51.329758 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bcb3e6-6a6a-43ef-8af8-ae7187c6bdf6 None admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:51.330381 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d63537-7da0-4bf3-b0de-8a3c5af5e05f None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:51.330478 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bcb3e6-6a6a-43ef-8af8-ae7187c6bdf6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:49:51.330669 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d63537-7da0-4bf3-b0de-8a3c5af5e05f None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:51.330760 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bcb3e6-6a6a-43ef-8af8-ae7187c6bdf6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:51.330940 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d63537-7da0-4bf3-b0de-8a3c5af5e05f None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:51.332350 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 56/109] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:49:51 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:49:51.332485 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bcb3e6-6a6a-43ef-8af8-ae7187c6bdf6 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:51.337281 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d63537-7da0-4bf3-b0de-8a3c5af5e05f None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:51.341856 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 54/110] 10.222.0.18 () {56 vars in 1099 bytes} [Sat Aug 6 21:49:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:49:51.341856 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de24332-b202-4062-bf93-14c64c6af962 None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:51.341856 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de24332-b202-4062-bf93-14c64c6af962 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:51.341856 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de24332-b202-4062-bf93-14c64c6af962 None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:51.341856 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 55/111] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:49:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:49:51.347080 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d54ef-2eb8-40de-ae72-2ee1a10e9582 None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:51.347271 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d54ef-2eb8-40de-ae72-2ee1a10e9582 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:51.347546 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d54ef-2eb8-40de-ae72-2ee1a10e9582 None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:51.348964 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 56/112] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:49:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:49:51.350066 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed50dad-db4a-4ff5-927a-0518e83fa30b None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:51.350356 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed50dad-db4a-4ff5-927a-0518e83fa30b None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:51.350631 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed50dad-db4a-4ff5-927a-0518e83fa30b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:51.379201 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-fed50dad-db4a-4ff5-927a-0518e83fa30b None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:49:51.380586 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-fed50dad-db4a-4ff5-927a-0518e83fa30b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:49:51.393948 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:51.464440 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 57/113] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:49:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 21:49:51.464440 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed50dad-db4a-4ff5-927a-0518e83fa30b None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:51.486320 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d63537-7da0-4bf3-b0de-8a3c5af5e05f None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:51.531526 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a6be1d-9413-4e5a-9076-4eede0999247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:51.532161 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a6be1d-9413-4e5a-9076-4eede0999247 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:51.532464 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a6be1d-9413-4e5a-9076-4eede0999247 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:51.532736 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a6be1d-9413-4e5a-9076-4eede0999247 None admin] PATH_INFO: `/v3/users/7e19a292a7d749c291e942bc238399ba` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:51.555333 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a6be1d-9413-4e5a-9076-4eede0999247 None admin] RBAC: Authorizing `identity:get_user(user_id=7e19a292a7d749c291e942bc238399ba)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:51.557501 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a6be1d-9413-4e5a-9076-4eede0999247 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:51.559225 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 57/114] 10.222.0.18 () {56 vars in 1177 bytes} [Sat Aug 6 21:49:51 2022] GET /identity/v3/users/7e19a292a7d749c291e942bc238399ba => generated 289 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:49:51.560379 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0e09e7-6448-4862-be5a-bfe2103c2275 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:51.560683 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0e09e7-6448-4862-be5a-bfe2103c2275 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:51.560954 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0e09e7-6448-4862-be5a-bfe2103c2275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:51.579051 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-ac0e09e7-6448-4862-be5a-bfe2103c2275 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:49:51.580362 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-ac0e09e7-6448-4862-be5a-bfe2103c2275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:49:51.584010 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed50dad-db4a-4ff5-927a-0518e83fa30b None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:51.605078 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:51.622293 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbead087-5029-4060-bbd2-4a864f8a7391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:51.622933 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbead087-5029-4060-bbd2-4a864f8a7391 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:51.623215 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbead087-5029-4060-bbd2-4a864f8a7391 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:51.623488 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbead087-5029-4060-bbd2-4a864f8a7391 None admin] PATH_INFO: `/v3/users/6bb7ac25afcb4fcaba91aca891ca7fc9` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:51.624715 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbead087-5029-4060-bbd2-4a864f8a7391 None admin] RBAC: Authorizing `identity:get_user(user_id=6bb7ac25afcb4fcaba91aca891ca7fc9)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:51.626164 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbead087-5029-4060-bbd2-4a864f8a7391 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:51.627989 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 58/115] 10.222.0.18 () {56 vars in 1177 bytes} [Sat Aug 6 21:49:51 2022] GET /identity/v3/users/6bb7ac25afcb4fcaba91aca891ca7fc9 => generated 247 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:49:51.632456 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3a6343-99ed-4b82-8202-c77197ff75fd None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:51.632456 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3a6343-99ed-4b82-8202-c77197ff75fd None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:51.632456 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3a6343-99ed-4b82-8202-c77197ff75fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:51.649111 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-cf3a6343-99ed-4b82-8202-c77197ff75fd None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:49:51.650370 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-cf3a6343-99ed-4b82-8202-c77197ff75fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:49:51.661378 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:51.662437 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 58/116] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:49:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 21:49:51.663150 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0e09e7-6448-4862-be5a-bfe2103c2275 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:51.732883 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 59/117] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:49:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 21:49:51.733483 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3a6343-99ed-4b82-8202-c77197ff75fd None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:51.783078 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0e09e7-6448-4862-be5a-bfe2103c2275 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:51.821166 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5b3613-3f55-4dbb-b4a7-bf9f38bf016f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:51.821930 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5b3613-3f55-4dbb-b4a7-bf9f38bf016f None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:51.822214 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5b3613-3f55-4dbb-b4a7-bf9f38bf016f None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:51.822570 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5b3613-3f55-4dbb-b4a7-bf9f38bf016f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:51.834601 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5b3613-3f55-4dbb-b4a7-bf9f38bf016f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:49:51.839297 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-cf5b3613-3f55-4dbb-b4a7-bf9f38bf016f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 21:49:51.839297 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:49:51.839297 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:49:51.839297 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:49:51.839297 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:49:51.839297 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:49:51.839297 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:49:51.839297 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:49:51.839297 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:49:51.839297 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:49:51.839297 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:49:51.839297 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:49:51.839297 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:49:51.839297 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:49:51.839297 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:49:51.839297 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:49:51.840161 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:49:51.840161 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:49:51.840161 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:49:51.840161 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:49:51.840161 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:49:51.840161 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:49:51.840161 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:49:51.840161 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:49:51.840161 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:49:51.840161 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:49:51.840161 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:49:51.840161 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:49:51.840161 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:49:51.840161 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:49:51.840161 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:49:51.840161 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:49:51.841264 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:49:51.841264 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:49:51.841264 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:49:51.841264 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:49:51.841264 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:49:51.841264 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:49:51.841264 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:49:51.841264 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:49:51.841264 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:49:51.841264 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:49:51.841264 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 21:49:51.841264 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:49:51.841264 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 59/118] 10.222.0.18 () {56 vars in 1099 bytes} [Sat Aug 6 21:49:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:49:51.841937 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3a6343-99ed-4b82-8202-c77197ff75fd None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:51.845823 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5b3613-3f55-4dbb-b4a7-bf9f38bf016f None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:51.864779 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:51.879254 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437b327c-3e79-46a0-99e0-a595077fff6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:51.879966 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437b327c-3e79-46a0-99e0-a595077fff6f None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:51.880247 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437b327c-3e79-46a0-99e0-a595077fff6f None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:51.880553 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437b327c-3e79-46a0-99e0-a595077fff6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:51.896126 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:51.954365 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5b3613-3f55-4dbb-b4a7-bf9f38bf016f None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:51.958717 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437b327c-3e79-46a0-99e0-a595077fff6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:51.960368 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437b327c-3e79-46a0-99e0-a595077fff6f None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:51.997546 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90501022-8789-4b82-aeea-d94e5a7feacc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:51.998100 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90501022-8789-4b82-aeea-d94e5a7feacc None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:51.998381 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90501022-8789-4b82-aeea-d94e5a7feacc None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:51.998646 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90501022-8789-4b82-aeea-d94e5a7feacc None admin] PATH_INFO: `/v3/projects/edee8005e8714a9db1c10cfec12857b6` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:51.999769 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90501022-8789-4b82-aeea-d94e5a7feacc None admin] RBAC: Authorizing `identity:get_project(project_id=edee8005e8714a9db1c10cfec12857b6)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:52.001412 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90501022-8789-4b82-aeea-d94e5a7feacc None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:52.003197 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 60/119] 10.222.0.18 () {56 vars in 1186 bytes} [Sat Aug 6 21:49:51 2022] GET /identity/v3/projects/edee8005e8714a9db1c10cfec12857b6 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:49:52.003840 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90501022-8789-4b82-aeea-d94e5a7feacc None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:52.075886 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 60/120] 10.222.0.18 () {58 vars in 1282 bytes} [Sat Aug 6 21:49:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 21:49:52.077142 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437b327c-3e79-46a0-99e0-a595077fff6f None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:52.099209 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:52.116295 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90501022-8789-4b82-aeea-d94e5a7feacc None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:52.129499 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:52.153629 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ffff95-4497-45e9-a6b0-03dd529e8dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:52.154852 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ffff95-4497-45e9-a6b0-03dd529e8dc5 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:52.154852 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ffff95-4497-45e9-a6b0-03dd529e8dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:52.154852 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ffff95-4497-45e9-a6b0-03dd529e8dc5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:52.165410 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ffff95-4497-45e9-a6b0-03dd529e8dc5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:49:52.170211 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-05ffff95-4497-45e9-a6b0-03dd529e8dc5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 21:49:52.170211 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:49:52.170211 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:49:52.170211 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:49:52.170211 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:49:52.170211 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:49:52.170211 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:49:52.170211 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:49:52.170211 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:49:52.170211 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:49:52.170211 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:49:52.170211 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:49:52.170211 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:49:52.170211 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:49:52.170211 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:49:52.170211 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:49:52.172015 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:49:52.172015 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:49:52.172015 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:49:52.172015 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:49:52.172015 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:49:52.172015 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:49:52.172015 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:49:52.172015 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:49:52.172015 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:49:52.172015 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:49:52.172015 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:49:52.172015 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:49:52.172015 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:49:52.172015 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:49:52.172015 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:49:52.172015 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:49:52.173013 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:49:52.173013 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:49:52.173013 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:49:52.173013 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:49:52.173013 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:49:52.173013 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:49:52.173013 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:49:52.173013 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:49:52.173013 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:49:52.173013 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:49:52.173013 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 21:49:52.173013 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:49:52.173013 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 61/121] 10.222.0.18 () {56 vars in 1114 bytes} [Sat Aug 6 21:49:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:49:52.173013 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ffff95-4497-45e9-a6b0-03dd529e8dc5 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:52.186825 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437b327c-3e79-46a0-99e0-a595077fff6f None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:52.224568 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784d8531-286c-4944-8894-47e80acc1c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:52.225333 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d8531-286c-4944-8894-47e80acc1c43 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:52.225727 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d8531-286c-4944-8894-47e80acc1c43 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:52.226013 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d8531-286c-4944-8894-47e80acc1c43 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:52.236575 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784d8531-286c-4944-8894-47e80acc1c43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:49:52.242028 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-784d8531-286c-4944-8894-47e80acc1c43 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 21:49:52.242028 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:49:52.242028 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:49:52.242028 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:49:52.242028 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:49:52.242028 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:49:52.242028 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:49:52.242028 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:49:52.242028 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:49:52.242028 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:49:52.242028 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:49:52.242028 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:49:52.242028 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:49:52.242028 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:49:52.242028 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:49:52.242028 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:49:52.243061 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:49:52.243061 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:49:52.243061 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:49:52.243061 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:49:52.243061 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:49:52.243061 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:49:52.243061 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:49:52.243061 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:49:52.243061 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:49:52.243061 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:49:52.243061 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:49:52.243061 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:49:52.243061 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:49:52.243061 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:49:52.243061 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:49:52.243061 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:49:52.244061 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:49:52.244061 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:49:52.244061 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:49:52.244061 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:49:52.244061 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:49:52.244061 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:49:52.244061 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:49:52.244061 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:49:52.244061 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:49:52.244061 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:49:52.244061 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 21:49:52.244061 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:49:52.245129 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 61/122] 10.222.0.18 () {56 vars in 1099 bytes} [Sat Aug 6 21:49:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:49:52.245834 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784d8531-286c-4944-8894-47e80acc1c43 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:52.292497 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ffff95-4497-45e9-a6b0-03dd529e8dc5 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:52.325971 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014d3689-4f11-4cb7-b721-2a8ceb65ca35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:52.326572 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d3689-4f11-4cb7-b721-2a8ceb65ca35 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:52.326860 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d3689-4f11-4cb7-b721-2a8ceb65ca35 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:52.327132 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d3689-4f11-4cb7-b721-2a8ceb65ca35 None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:52.327797 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014d3689-4f11-4cb7-b721-2a8ceb65ca35 None admin] RBAC: Adding query filter params (name=member) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:49:52.328099 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014d3689-4f11-4cb7-b721-2a8ceb65ca35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:52.329523 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014d3689-4f11-4cb7-b721-2a8ceb65ca35 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:52.338352 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 62/123] 10.222.0.18 () {56 vars in 1101 bytes} [Sat Aug 6 21:49:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:49:52.343541 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014d3689-4f11-4cb7-b721-2a8ceb65ca35 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:52.352799 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:52.353225 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784d8531-286c-4944-8894-47e80acc1c43 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:52.356017 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:52.391688 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dfe986-9a4f-4fb1-907b-9f4f77981ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:52.392525 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dfe986-9a4f-4fb1-907b-9f4f77981ea3 None admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:52.393169 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dfe986-9a4f-4fb1-907b-9f4f77981ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:52.393169 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dfe986-9a4f-4fb1-907b-9f4f77981ea3 None admin] PATH_INFO: `/v3/projects/edee8005e8714a9db1c10cfec12857b6/users/6bb7ac25afcb4fcaba91aca891ca7fc9/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:52.395827 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dfe986-9a4f-4fb1-907b-9f4f77981ea3 None admin] RBAC: Authorizing `identity:create_grant(project_id=edee8005e8714a9db1c10cfec12857b6, user_id=6bb7ac25afcb4fcaba91aca891ca7fc9, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:52.397357 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dfe986-9a4f-4fb1-907b-9f4f77981ea3 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:52.450894 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014d3689-4f11-4cb7-b721-2a8ceb65ca35 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:52.468445 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 62/124] 10.222.0.18 () {58 vars in 1439 bytes} [Sat Aug 6 21:49:52 2022] PUT /identity/v3/projects/edee8005e8714a9db1c10cfec12857b6/users/6bb7ac25afcb4fcaba91aca891ca7fc9/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 21:49:52.469449 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dfe986-9a4f-4fb1-907b-9f4f77981ea3 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:52.500506 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf323b9f-f53e-46d2-8046-317756ab4c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:52.501551 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf323b9f-f53e-46d2-8046-317756ab4c6c None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:52.502058 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf323b9f-f53e-46d2-8046-317756ab4c6c None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:52.502667 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf323b9f-f53e-46d2-8046-317756ab4c6c None admin] PATH_INFO: `/v3/projects/edee8005e8714a9db1c10cfec12857b6` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:52.505226 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf323b9f-f53e-46d2-8046-317756ab4c6c None admin] RBAC: Authorizing `identity:get_project(project_id=edee8005e8714a9db1c10cfec12857b6)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:52.507385 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf323b9f-f53e-46d2-8046-317756ab4c6c None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:52.509794 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 63/125] 10.222.0.18 () {56 vars in 1186 bytes} [Sat Aug 6 21:49:52 2022] GET /identity/v3/projects/edee8005e8714a9db1c10cfec12857b6 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:49:52.511183 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf323b9f-f53e-46d2-8046-317756ab4c6c None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:52.591254 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dfe986-9a4f-4fb1-907b-9f4f77981ea3 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:52.605658 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:52.631415 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165a642a-a86c-4e11-a8ee-06a3f92e4dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:52.632047 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165a642a-a86c-4e11-a8ee-06a3f92e4dca None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:52.632323 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165a642a-a86c-4e11-a8ee-06a3f92e4dca None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:52.632623 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165a642a-a86c-4e11-a8ee-06a3f92e4dca None admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:52.633304 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165a642a-a86c-4e11-a8ee-06a3f92e4dca None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:49:52.633607 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165a642a-a86c-4e11-a8ee-06a3f92e4dca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:52.635021 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165a642a-a86c-4e11-a8ee-06a3f92e4dca None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:52.637557 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf323b9f-f53e-46d2-8046-317756ab4c6c None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:52.647266 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 63/126] 10.222.0.18 () {56 vars in 1111 bytes} [Sat Aug 6 21:49:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:49:52.647853 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165a642a-a86c-4e11-a8ee-06a3f92e4dca None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:52.680543 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edfbec8-dc39-4e30-b90b-14aa487e7ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:52.681584 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edfbec8-dc39-4e30-b90b-14aa487e7ddf None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:52.682171 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edfbec8-dc39-4e30-b90b-14aa487e7ddf None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:52.682553 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edfbec8-dc39-4e30-b90b-14aa487e7ddf None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:52.683332 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edfbec8-dc39-4e30-b90b-14aa487e7ddf None admin] RBAC: Adding query filter params (name=member) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:49:52.683757 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edfbec8-dc39-4e30-b90b-14aa487e7ddf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:52.685714 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edfbec8-dc39-4e30-b90b-14aa487e7ddf None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:52.688253 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:52.696492 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 64/127] 10.222.0.18 () {56 vars in 1101 bytes} [Sat Aug 6 21:49:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:49:52.699706 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edfbec8-dc39-4e30-b90b-14aa487e7ddf None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:52.783549 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165a642a-a86c-4e11-a8ee-06a3f92e4dca None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:52.813293 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edfbec8-dc39-4e30-b90b-14aa487e7ddf None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:52.830350 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bfe408-c48c-4b8e-a525-5fd835617ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:52.831288 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bfe408-c48c-4b8e-a525-5fd835617ae3 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:52.832175 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bfe408-c48c-4b8e-a525-5fd835617ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:52.832482 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bfe408-c48c-4b8e-a525-5fd835617ae3 None admin] PATH_INFO: `/v3/users/b7b3753a7de94d39b55d2106b904b48d` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:52.833820 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bfe408-c48c-4b8e-a525-5fd835617ae3 None admin] RBAC: Authorizing `identity:get_user(user_id=b7b3753a7de94d39b55d2106b904b48d)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:52.836043 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bfe408-c48c-4b8e-a525-5fd835617ae3 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:52.838399 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 64/128] 10.222.0.18 () {56 vars in 1177 bytes} [Sat Aug 6 21:49:52 2022] GET /identity/v3/users/b7b3753a7de94d39b55d2106b904b48d => generated 275 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:49:52.840959 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bfe408-c48c-4b8e-a525-5fd835617ae3 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:52.854684 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdcfd81-609c-407f-85f0-515f6fbc7d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:52.855273 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdcfd81-609c-407f-85f0-515f6fbc7d9c None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:52.855555 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdcfd81-609c-407f-85f0-515f6fbc7d9c None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:52.855908 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdcfd81-609c-407f-85f0-515f6fbc7d9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:52.856586 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdcfd81-609c-407f-85f0-515f6fbc7d9c None admin] RBAC: Adding query filter params (role.id=817eabff1ca5470993116be35777000a, scope.project.id=edee8005e8714a9db1c10cfec12857b6, user.id=7e19a292a7d749c291e942bc238399ba) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:49:52.856897 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdcfd81-609c-407f-85f0-515f6fbc7d9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:52.858676 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdcfd81-609c-407f-85f0-515f6fbc7d9c None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:52.861618 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:52.865086 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 65/129] 10.222.0.18 () {56 vars in 1374 bytes} [Sat Aug 6 21:49:52 2022] GET /identity/v3/role_assignments?user.id=7e19a292a7d749c291e942bc238399ba&scope.project.id=edee8005e8714a9db1c10cfec12857b6&role.id=817eabff1ca5470993116be35777000a => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:49:52.866268 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdcfd81-609c-407f-85f0-515f6fbc7d9c None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:52.985828 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdcfd81-609c-407f-85f0-515f6fbc7d9c None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:52.986440 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:52.995560 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bfe408-c48c-4b8e-a525-5fd835617ae3 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:53.025441 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0cd420-8145-463a-ad0a-a6aacf23929c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:53.026455 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0cd420-8145-463a-ad0a-a6aacf23929c None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:53.026929 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0cd420-8145-463a-ad0a-a6aacf23929c None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:53.027393 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0cd420-8145-463a-ad0a-a6aacf23929c None admin] PATH_INFO: `/v3/users/b7b3753a7de94d39b55d2106b904b48d` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:53.028899 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0cd420-8145-463a-ad0a-a6aacf23929c None admin] RBAC: Authorizing `identity:get_user(user_id=b7b3753a7de94d39b55d2106b904b48d)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:53.030709 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0cd420-8145-463a-ad0a-a6aacf23929c None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:53.032637 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 66/130] 10.222.0.18 () {56 vars in 1177 bytes} [Sat Aug 6 21:49:52 2022] GET /identity/v3/users/b7b3753a7de94d39b55d2106b904b48d => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:49:53.033735 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0cd420-8145-463a-ad0a-a6aacf23929c None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:53.050887 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2850fe19-c5cd-43d8-be01-d455ef632720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:53.051950 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2850fe19-c5cd-43d8-be01-d455ef632720 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:53.052448 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2850fe19-c5cd-43d8-be01-d455ef632720 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:53.052944 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2850fe19-c5cd-43d8-be01-d455ef632720 None admin] PATH_INFO: `/v3/users/b7b3753a7de94d39b55d2106b904b48d` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:53.054455 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2850fe19-c5cd-43d8-be01-d455ef632720 None admin] RBAC: Authorizing `identity:get_user(user_id=b7b3753a7de94d39b55d2106b904b48d)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:53.056367 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2850fe19-c5cd-43d8-be01-d455ef632720 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:53.058321 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 65/131] 10.222.0.18 () {56 vars in 1177 bytes} [Sat Aug 6 21:49:52 2022] GET /identity/v3/users/b7b3753a7de94d39b55d2106b904b48d => generated 275 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:49:53.059452 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2850fe19-c5cd-43d8-be01-d455ef632720 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:53.159020 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:53.176954 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2850fe19-c5cd-43d8-be01-d455ef632720 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:53.196341 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0cd420-8145-463a-ad0a-a6aacf23929c None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:53.229127 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269d2dd9-c8e5-43b3-924a-a98054bde689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:53.229784 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269d2dd9-c8e5-43b3-924a-a98054bde689 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:53.230065 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269d2dd9-c8e5-43b3-924a-a98054bde689 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:53.230412 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269d2dd9-c8e5-43b3-924a-a98054bde689 None admin] PATH_INFO: `/v3/projects/f89b12d09e6449f990095cb05e6b5627` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:53.231556 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269d2dd9-c8e5-43b3-924a-a98054bde689 None admin] RBAC: Authorizing `identity:get_project(project_id=f89b12d09e6449f990095cb05e6b5627)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:53.233623 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269d2dd9-c8e5-43b3-924a-a98054bde689 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:53.235555 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 66/132] 10.222.0.18 () {56 vars in 1186 bytes} [Sat Aug 6 21:49:53 2022] GET /identity/v3/projects/f89b12d09e6449f990095cb05e6b5627 => generated 312 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:49:53.235758 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9906b4-72f7-4e95-a611-a5e437f03027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:53.236154 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269d2dd9-c8e5-43b3-924a-a98054bde689 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:53.236978 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9906b4-72f7-4e95-a611-a5e437f03027 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:53.237513 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9906b4-72f7-4e95-a611-a5e437f03027 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:53.238832 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9906b4-72f7-4e95-a611-a5e437f03027 None admin] PATH_INFO: `/v3/projects/edee8005e8714a9db1c10cfec12857b6` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:53.240243 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9906b4-72f7-4e95-a611-a5e437f03027 None admin] RBAC: Authorizing `identity:get_project(project_id=edee8005e8714a9db1c10cfec12857b6)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:53.242117 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9906b4-72f7-4e95-a611-a5e437f03027 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:53.244253 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 67/133] 10.222.0.18 () {56 vars in 1186 bytes} [Sat Aug 6 21:49:53 2022] GET /identity/v3/projects/edee8005e8714a9db1c10cfec12857b6 => generated 298 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:49:53.245217 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9906b4-72f7-4e95-a611-a5e437f03027 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:53.294526 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:53.354732 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9906b4-72f7-4e95-a611-a5e437f03027 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:53.389006 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269d2dd9-c8e5-43b3-924a-a98054bde689 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:53.397242 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0315d8ca-5f0c-4fb2-844f-4554c10bc958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:53.398243 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0315d8ca-5f0c-4fb2-844f-4554c10bc958 None admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:53.398740 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0315d8ca-5f0c-4fb2-844f-4554c10bc958 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:53.399203 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0315d8ca-5f0c-4fb2-844f-4554c10bc958 None admin] PATH_INFO: `/v3/projects/f89b12d09e6449f990095cb05e6b5627/users/b7b3753a7de94d39b55d2106b904b48d/roles/56e4a96094e9434cbbafb22579c034f5` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:53.404888 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0315d8ca-5f0c-4fb2-844f-4554c10bc958 None admin] RBAC: Authorizing `identity:create_grant(project_id=f89b12d09e6449f990095cb05e6b5627, user_id=b7b3753a7de94d39b55d2106b904b48d, role_id=56e4a96094e9434cbbafb22579c034f5)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:53.406612 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0315d8ca-5f0c-4fb2-844f-4554c10bc958 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:53.414569 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:53.439258 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4b5f13-3dfd-4553-9f40-363e6c80723b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:53.440274 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4b5f13-3dfd-4553-9f40-363e6c80723b None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:53.441407 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4b5f13-3dfd-4553-9f40-363e6c80723b None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:53.441903 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4b5f13-3dfd-4553-9f40-363e6c80723b None admin] PATH_INFO: `/v3/projects/edee8005e8714a9db1c10cfec12857b6` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:53.443253 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4b5f13-3dfd-4553-9f40-363e6c80723b None admin] RBAC: Authorizing `identity:get_project(project_id=edee8005e8714a9db1c10cfec12857b6)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:53.445017 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4b5f13-3dfd-4553-9f40-363e6c80723b None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:53.447223 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 67/134] 10.222.0.18 () {56 vars in 1186 bytes} [Sat Aug 6 21:49:53 2022] GET /identity/v3/projects/edee8005e8714a9db1c10cfec12857b6 => generated 298 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:49:53.448187 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4b5f13-3dfd-4553-9f40-363e6c80723b None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:53.495246 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 68/135] 10.222.0.18 () {58 vars in 1439 bytes} [Sat Aug 6 21:49:53 2022] PUT /identity/v3/projects/f89b12d09e6449f990095cb05e6b5627/users/b7b3753a7de94d39b55d2106b904b48d/roles/56e4a96094e9434cbbafb22579c034f5 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 21:49:53.496451 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0315d8ca-5f0c-4fb2-844f-4554c10bc958 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:53.621062 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:53.626930 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4b5f13-3dfd-4553-9f40-363e6c80723b None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:53.687754 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4b22a9-f216-4b71-aa91-7dba5c6df269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:53.688368 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b22a9-f216-4b71-aa91-7dba5c6df269 None admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:53.688675 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b22a9-f216-4b71-aa91-7dba5c6df269 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:53.688955 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b22a9-f216-4b71-aa91-7dba5c6df269 None admin] PATH_INFO: `/v3/projects/edee8005e8714a9db1c10cfec12857b6/users/b7b3753a7de94d39b55d2106b904b48d/roles/56e4a96094e9434cbbafb22579c034f5` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:53.694411 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0315d8ca-5f0c-4fb2-844f-4554c10bc958 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:53.697500 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4b22a9-f216-4b71-aa91-7dba5c6df269 None admin] RBAC: Authorizing `identity:create_grant(project_id=edee8005e8714a9db1c10cfec12857b6, user_id=b7b3753a7de94d39b55d2106b904b48d, role_id=56e4a96094e9434cbbafb22579c034f5)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:53.698997 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4b22a9-f216-4b71-aa91-7dba5c6df269 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:53.739238 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02629f25-8553-4be6-bdf3-b8ef31c4c579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:53.739903 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02629f25-8553-4be6-bdf3-b8ef31c4c579 None admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:53.740189 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02629f25-8553-4be6-bdf3-b8ef31c4c579 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:53.740503 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02629f25-8553-4be6-bdf3-b8ef31c4c579 None admin] PATH_INFO: `/v3/projects/edee8005e8714a9db1c10cfec12857b6/users/b7b3753a7de94d39b55d2106b904b48d/roles/93c058ac49e242fd8517c80c552dea2d` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:53.742679 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02629f25-8553-4be6-bdf3-b8ef31c4c579 None admin] RBAC: Authorizing `identity:create_grant(project_id=edee8005e8714a9db1c10cfec12857b6, user_id=b7b3753a7de94d39b55d2106b904b48d, role_id=93c058ac49e242fd8517c80c552dea2d)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:53.744131 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02629f25-8553-4be6-bdf3-b8ef31c4c579 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:53.781237 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 68/136] 10.222.0.18 () {58 vars in 1439 bytes} [Sat Aug 6 21:49:53 2022] PUT /identity/v3/projects/edee8005e8714a9db1c10cfec12857b6/users/b7b3753a7de94d39b55d2106b904b48d/roles/56e4a96094e9434cbbafb22579c034f5 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 21:49:53.782303 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eab58b-4fa2-4feb-a498-d2565f7d095b None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:53.782590 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eab58b-4fa2-4feb-a498-d2565f7d095b None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:53.782865 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eab58b-4fa2-4feb-a498-d2565f7d095b None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:53.784210 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 69/137] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:49:53 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:49:53.790697 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9499625-9ca4-4e66-9dad-1615ac161854 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:53.822908 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9499625-9ca4-4e66-9dad-1615ac161854 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:53.822908 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9499625-9ca4-4e66-9dad-1615ac161854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:53.822908 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-e9499625-9ca4-4e66-9dad-1615ac161854 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:49:53.822908 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-e9499625-9ca4-4e66-9dad-1615ac161854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:49:53.902978 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 69/138] 10.222.0.18 () {58 vars in 1439 bytes} [Sat Aug 6 21:49:53 2022] PUT /identity/v3/projects/edee8005e8714a9db1c10cfec12857b6/users/b7b3753a7de94d39b55d2106b904b48d/roles/93c058ac49e242fd8517c80c552dea2d => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 21:49:53.924953 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 70/139] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:49:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 21:49:53.970051 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02629f25-8553-4be6-bdf3-b8ef31c4c579 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:54.051161 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:54.101393 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02629f25-8553-4be6-bdf3-b8ef31c4c579 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:54.140602 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de03d316-f1b9-4d0c-bae9-1004658a10ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:54.141254 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de03d316-f1b9-4d0c-bae9-1004658a10ae None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:54.141540 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de03d316-f1b9-4d0c-bae9-1004658a10ae None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:54.141817 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de03d316-f1b9-4d0c-bae9-1004658a10ae None admin] PATH_INFO: `/v3/domains/default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:54.142953 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de03d316-f1b9-4d0c-bae9-1004658a10ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:54.145216 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de03d316-f1b9-4d0c-bae9-1004658a10ae None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:54.146990 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 70/140] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:49:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:49:54.152751 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9499625-9ca4-4e66-9dad-1615ac161854 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:54.259007 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:54.276250 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9499625-9ca4-4e66-9dad-1615ac161854 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:54.323542 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81947a8f-c350-40d8-9afc-91d36fb3a78c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:54.324178 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81947a8f-c350-40d8-9afc-91d36fb3a78c None admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:54.324552 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81947a8f-c350-40d8-9afc-91d36fb3a78c None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:54.324832 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81947a8f-c350-40d8-9afc-91d36fb3a78c None admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:54.325498 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81947a8f-c350-40d8-9afc-91d36fb3a78c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:54.326959 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81947a8f-c350-40d8-9afc-91d36fb3a78c None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:54.455207 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 71/141] 10.222.0.18 () {60 vars in 1132 bytes} [Sat Aug 6 21:49:54 2022] POST /identity/v3/users => generated 283 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 21:49:55.228435 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2c355c-0fd2-47b6-82fb-b75007703cb2 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:55.228956 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2c355c-0fd2-47b6-82fb-b75007703cb2 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:55.229227 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2c355c-0fd2-47b6-82fb-b75007703cb2 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:55.230609 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 71/142] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:49:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:49:55.236227 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c826949c-1209-4a32-b62b-79f41f66a6fb None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:55.236782 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c826949c-1209-4a32-b62b-79f41f66a6fb None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:55.237262 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c826949c-1209-4a32-b62b-79f41f66a6fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:55.261435 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-c826949c-1209-4a32-b62b-79f41f66a6fb None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:49:55.262946 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-c826949c-1209-4a32-b62b-79f41f66a6fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:49:55.318114 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb7b788-0f4c-4121-8d14-c020647f97fb None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:55.318729 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb7b788-0f4c-4121-8d14-c020647f97fb None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:55.319199 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb7b788-0f4c-4121-8d14-c020647f97fb None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:55.320763 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 72/143] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:49:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 21:49:55.327607 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a51ac-aed8-4204-8063-a16aad8b18d3 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:55.328161 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a51ac-aed8-4204-8063-a16aad8b18d3 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:55.328682 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a51ac-aed8-4204-8063-a16aad8b18d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:55.356884 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-6e3a51ac-aed8-4204-8063-a16aad8b18d3 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:49:55.358590 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-6e3a51ac-aed8-4204-8063-a16aad8b18d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:49:55.373498 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 72/144] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:49:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 21:49:55.426902 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c826949c-1209-4a32-b62b-79f41f66a6fb None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:55.463492 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 73/145] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:49:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 21:49:55.493757 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3a51ac-aed8-4204-8063-a16aad8b18d3 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:55.540143 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:55.564710 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:55.586483 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c826949c-1209-4a32-b62b-79f41f66a6fb None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:55.624034 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3a51ac-aed8-4204-8063-a16aad8b18d3 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:55.653798 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7771e8-f020-4e8c-b39c-3f40c94cba44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:55.654444 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7771e8-f020-4e8c-b39c-3f40c94cba44 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:55.654730 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7771e8-f020-4e8c-b39c-3f40c94cba44 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:55.655004 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7771e8-f020-4e8c-b39c-3f40c94cba44 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:55.678390 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7771e8-f020-4e8c-b39c-3f40c94cba44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:49:55.683307 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-3d7771e8-f020-4e8c-b39c-3f40c94cba44 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 21:49:55.683307 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:49:55.683307 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:49:55.683307 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:49:55.683307 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:49:55.683307 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:49:55.683307 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:49:55.683307 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:49:55.683307 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:49:55.683307 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:49:55.683307 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:49:55.683307 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:49:55.683307 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:49:55.683307 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:49:55.683307 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:49:55.683307 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:49:55.684185 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:49:55.684185 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:49:55.684185 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:49:55.684185 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:49:55.684185 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:49:55.684185 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:49:55.684185 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:49:55.684185 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:49:55.684185 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:49:55.684185 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:49:55.684185 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:49:55.684185 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:49:55.684185 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:49:55.684185 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:49:55.684185 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:49:55.684185 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:49:55.685009 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:49:55.685009 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:49:55.685009 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:49:55.685009 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:49:55.685009 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:49:55.685009 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:49:55.685009 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:49:55.685009 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:49:55.685009 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:49:55.685009 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:49:55.685009 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 21:49:55.685009 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:49:55.685009 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 73/146] 10.222.0.18 () {56 vars in 1096 bytes} [Sat Aug 6 21:49:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 21:49:55.693711 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7771e8-f020-4e8c-b39c-3f40c94cba44 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:55.710004 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186a8843-0ed6-42b6-b6d2-ed23247a701b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:55.710760 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186a8843-0ed6-42b6-b6d2-ed23247a701b None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:55.711051 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186a8843-0ed6-42b6-b6d2-ed23247a701b None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:55.711330 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186a8843-0ed6-42b6-b6d2-ed23247a701b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:55.732888 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186a8843-0ed6-42b6-b6d2-ed23247a701b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:49:55.738519 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-186a8843-0ed6-42b6-b6d2-ed23247a701b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 21:49:55.738519 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:49:55.738519 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:49:55.738519 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:49:55.738519 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:49:55.738519 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:49:55.738519 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:49:55.738519 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:49:55.738519 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:49:55.738519 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:49:55.738519 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:49:55.738519 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:49:55.738519 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:49:55.738519 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:49:55.738519 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:49:55.738519 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:49:55.739365 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:49:55.739365 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:49:55.739365 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:49:55.739365 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:49:55.739365 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:49:55.739365 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:49:55.739365 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:49:55.739365 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:49:55.739365 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:49:55.739365 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:49:55.739365 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:49:55.739365 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:49:55.739365 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:49:55.739365 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:49:55.739365 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:49:55.739365 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:49:55.740114 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:49:55.740114 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:49:55.740114 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:49:55.740114 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:49:55.740114 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:49:55.740114 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:49:55.740114 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:49:55.740114 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:49:55.740114 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:49:55.740114 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:49:55.740114 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 21:49:55.740114 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:49:55.740114 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 74/147] 10.222.0.18 () {56 vars in 1099 bytes} [Sat Aug 6 21:49:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:49:55.763149 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186a8843-0ed6-42b6-b6d2-ed23247a701b None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:55.823373 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7771e8-f020-4e8c-b39c-3f40c94cba44 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:55.860681 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b098055-c0d5-48b1-87c5-a273d6a0ebec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:55.861779 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b098055-c0d5-48b1-87c5-a273d6a0ebec None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:55.862280 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b098055-c0d5-48b1-87c5-a273d6a0ebec None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:55.863283 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b098055-c0d5-48b1-87c5-a273d6a0ebec None admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:55.868430 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b098055-c0d5-48b1-87c5-a273d6a0ebec None admin] RBAC: Adding query filter params (name=admin) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:49:55.869788 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b098055-c0d5-48b1-87c5-a273d6a0ebec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:55.871273 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b098055-c0d5-48b1-87c5-a273d6a0ebec None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:55.884804 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:55.901236 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 74/148] 10.222.0.18 () {56 vars in 1099 bytes} [Sat Aug 6 21:49:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:49:55.908366 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b098055-c0d5-48b1-87c5-a273d6a0ebec None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:55.956369 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186a8843-0ed6-42b6-b6d2-ed23247a701b None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:55.985748 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:56.024427 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6693160c-8ec1-447a-9079-b4ec7ade376e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:56.025369 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6693160c-8ec1-447a-9079-b4ec7ade376e None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:56.025838 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6693160c-8ec1-447a-9079-b4ec7ade376e None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:56.026352 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6693160c-8ec1-447a-9079-b4ec7ade376e None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:56.027224 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6693160c-8ec1-447a-9079-b4ec7ade376e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:49:56.027719 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6693160c-8ec1-447a-9079-b4ec7ade376e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:56.029330 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6693160c-8ec1-447a-9079-b4ec7ade376e None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:56.041895 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 75/149] 10.222.0.18 () {56 vars in 1101 bytes} [Sat Aug 6 21:49:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:49:56.048009 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6693160c-8ec1-447a-9079-b4ec7ade376e None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:56.089807 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b098055-c0d5-48b1-87c5-a273d6a0ebec None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:56.130754 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7583d0f1-bd4b-4fda-90ab-1c21f32d2cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:56.131362 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7583d0f1-bd4b-4fda-90ab-1c21f32d2cf7 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:56.131628 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7583d0f1-bd4b-4fda-90ab-1c21f32d2cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:56.131985 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7583d0f1-bd4b-4fda-90ab-1c21f32d2cf7 None admin] PATH_INFO: `/v3/users/6bb7ac25afcb4fcaba91aca891ca7fc9` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:56.137186 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7583d0f1-bd4b-4fda-90ab-1c21f32d2cf7 None admin] RBAC: Authorizing `identity:get_user(user_id=6bb7ac25afcb4fcaba91aca891ca7fc9)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:56.138789 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7583d0f1-bd4b-4fda-90ab-1c21f32d2cf7 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:56.140813 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 75/150] 10.222.0.18 () {56 vars in 1177 bytes} [Sat Aug 6 21:49:55 2022] GET /identity/v3/users/6bb7ac25afcb4fcaba91aca891ca7fc9 => generated 247 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:49:56.154609 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7583d0f1-bd4b-4fda-90ab-1c21f32d2cf7 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:56.184079 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6693160c-8ec1-447a-9079-b4ec7ade376e None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:56.241155 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d70202-1fd4-46b8-b5c0-7d9b5238e459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:56.241883 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d70202-1fd4-46b8-b5c0-7d9b5238e459 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:56.242135 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d70202-1fd4-46b8-b5c0-7d9b5238e459 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:56.242414 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d70202-1fd4-46b8-b5c0-7d9b5238e459 None admin] PATH_INFO: `/v3/users/7e19a292a7d749c291e942bc238399ba` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:56.243660 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d70202-1fd4-46b8-b5c0-7d9b5238e459 None admin] RBAC: Authorizing `identity:get_user(user_id=7e19a292a7d749c291e942bc238399ba)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:56.245176 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d70202-1fd4-46b8-b5c0-7d9b5238e459 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:56.246806 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 76/151] 10.222.0.18 () {56 vars in 1177 bytes} [Sat Aug 6 21:49:56 2022] GET /identity/v3/users/7e19a292a7d749c291e942bc238399ba => generated 289 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:49:56.252520 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d70202-1fd4-46b8-b5c0-7d9b5238e459 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:56.261476 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:56.309396 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7583d0f1-bd4b-4fda-90ab-1c21f32d2cf7 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:56.367403 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c3a609-43b6-4113-8977-ff20e2e38ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:56.368033 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c3a609-43b6-4113-8977-ff20e2e38ff5 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:56.368312 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c3a609-43b6-4113-8977-ff20e2e38ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:56.368610 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c3a609-43b6-4113-8977-ff20e2e38ff5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:56.396768 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:56.472923 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c3a609-43b6-4113-8977-ff20e2e38ff5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:56.474403 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c3a609-43b6-4113-8977-ff20e2e38ff5 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:56.551206 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d70202-1fd4-46b8-b5c0-7d9b5238e459 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:56.566457 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:56.617808 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 76/152] 10.222.0.18 () {58 vars in 1282 bytes} [Sat Aug 6 21:49:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 06 21:49:56.623530 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c3a609-43b6-4113-8977-ff20e2e38ff5 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:56.656448 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dfe161-2218-47e7-bda7-b681ca8ed83c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:56.657101 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dfe161-2218-47e7-bda7-b681ca8ed83c None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:56.657384 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dfe161-2218-47e7-bda7-b681ca8ed83c None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:56.657663 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dfe161-2218-47e7-bda7-b681ca8ed83c None admin] PATH_INFO: `/v3/projects/edee8005e8714a9db1c10cfec12857b6` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:56.658805 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dfe161-2218-47e7-bda7-b681ca8ed83c None admin] RBAC: Authorizing `identity:get_project(project_id=edee8005e8714a9db1c10cfec12857b6)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:56.660433 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dfe161-2218-47e7-bda7-b681ca8ed83c None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:56.662114 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 77/153] 10.222.0.18 () {56 vars in 1186 bytes} [Sat Aug 6 21:49:56 2022] GET /identity/v3/projects/edee8005e8714a9db1c10cfec12857b6 => generated 298 bytes in 410 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:49:56.667990 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dfe161-2218-47e7-bda7-b681ca8ed83c None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:56.763855 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c3a609-43b6-4113-8977-ff20e2e38ff5 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:56.817879 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1dcef1-fc28-4304-93f0-6c735be604f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:56.818453 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1dcef1-fc28-4304-93f0-6c735be604f2 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:56.818733 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1dcef1-fc28-4304-93f0-6c735be604f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:56.819021 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1dcef1-fc28-4304-93f0-6c735be604f2 None admin] PATH_INFO: `/v3/projects/edee8005e8714a9db1c10cfec12857b6` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:56.829133 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1dcef1-fc28-4304-93f0-6c735be604f2 None admin] RBAC: Authorizing `identity:get_project(project_id=edee8005e8714a9db1c10cfec12857b6)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:56.829488 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dfe161-2218-47e7-bda7-b681ca8ed83c None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:56.830613 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1dcef1-fc28-4304-93f0-6c735be604f2 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:56.832806 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 77/154] 10.222.0.18 () {56 vars in 1186 bytes} [Sat Aug 6 21:49:56 2022] GET /identity/v3/projects/edee8005e8714a9db1c10cfec12857b6 => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:49:56.841402 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1dcef1-fc28-4304-93f0-6c735be604f2 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:56.847605 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:56.880134 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132674e2-f1a3-4041-9fae-64419f1f8d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:56.880840 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132674e2-f1a3-4041-9fae-64419f1f8d23 None admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:56.881122 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132674e2-f1a3-4041-9fae-64419f1f8d23 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:56.881398 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132674e2-f1a3-4041-9fae-64419f1f8d23 None admin] PATH_INFO: `/v3/projects/edee8005e8714a9db1c10cfec12857b6/users/7e19a292a7d749c291e942bc238399ba/roles/817eabff1ca5470993116be35777000a` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:56.889730 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132674e2-f1a3-4041-9fae-64419f1f8d23 None admin] RBAC: Authorizing `identity:create_grant(project_id=edee8005e8714a9db1c10cfec12857b6, user_id=7e19a292a7d749c291e942bc238399ba, role_id=817eabff1ca5470993116be35777000a)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:56.891202 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132674e2-f1a3-4041-9fae-64419f1f8d23 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:56.894420 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:56.971687 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1dcef1-fc28-4304-93f0-6c735be604f2 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:56.994629 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 78/155] 10.222.0.18 () {58 vars in 1439 bytes} [Sat Aug 6 21:49:56 2022] PUT /identity/v3/projects/edee8005e8714a9db1c10cfec12857b6/users/7e19a292a7d749c291e942bc238399ba/roles/817eabff1ca5470993116be35777000a => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 21:49:56.995959 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c2ee1b-f759-42bf-889b-f6dfc984052c None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:56.996330 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c2ee1b-f759-42bf-889b-f6dfc984052c None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:57.003856 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c2ee1b-f759-42bf-889b-f6dfc984052c None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:57.003856 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 79/156] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:49:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:49:57.005878 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb49a812-bf8f-45df-9926-cd386a7fff0e None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:57.006178 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb49a812-bf8f-45df-9926-cd386a7fff0e None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:57.006460 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb49a812-bf8f-45df-9926-cd386a7fff0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:57.016472 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a30ecc-c727-46c5-827c-783ba87fc94a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:57.017476 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a30ecc-c727-46c5-827c-783ba87fc94a None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:57.017848 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a30ecc-c727-46c5-827c-783ba87fc94a None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:57.018128 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a30ecc-c727-46c5-827c-783ba87fc94a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:57.019492 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a30ecc-c727-46c5-827c-783ba87fc94a None admin] RBAC: Adding query filter params (role.id=4b8dcf0491fd4072acc69fd894bf77d0, scope.project.id=edee8005e8714a9db1c10cfec12857b6, user.id=6bb7ac25afcb4fcaba91aca891ca7fc9) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:49:57.019809 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a30ecc-c727-46c5-827c-783ba87fc94a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:57.022293 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a30ecc-c727-46c5-827c-783ba87fc94a None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:57.027919 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 78/157] 10.222.0.18 () {56 vars in 1374 bytes} [Sat Aug 6 21:49:56 2022] GET /identity/v3/role_assignments?user.id=6bb7ac25afcb4fcaba91aca891ca7fc9&scope.project.id=edee8005e8714a9db1c10cfec12857b6&role.id=4b8dcf0491fd4072acc69fd894bf77d0 => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:49:57.044755 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-fb49a812-bf8f-45df-9926-cd386a7fff0e None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:49:57.046007 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-fb49a812-bf8f-45df-9926-cd386a7fff0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:49:57.154164 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 80/158] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:49:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 21:49:57.209353 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a30ecc-c727-46c5-827c-783ba87fc94a None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:57.328807 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:57.336737 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a30ecc-c727-46c5-827c-783ba87fc94a None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:57.376621 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec85295-ed75-464f-a12d-23b9efa9d92d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:57.377336 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec85295-ed75-464f-a12d-23b9efa9d92d None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:57.377617 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec85295-ed75-464f-a12d-23b9efa9d92d None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:57.377889 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec85295-ed75-464f-a12d-23b9efa9d92d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:57.390897 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec85295-ed75-464f-a12d-23b9efa9d92d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:49:57.396732 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-6ec85295-ed75-464f-a12d-23b9efa9d92d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 21:49:57.396732 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:49:57.396732 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:49:57.396732 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:49:57.396732 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:49:57.396732 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:49:57.396732 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:49:57.396732 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:49:57.396732 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:49:57.396732 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:49:57.396732 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:49:57.396732 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:49:57.396732 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:49:57.396732 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:49:57.396732 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:49:57.396732 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:49:57.397590 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:49:57.397590 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:49:57.397590 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:49:57.397590 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:49:57.397590 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:49:57.397590 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:49:57.397590 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:49:57.397590 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:49:57.397590 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:49:57.397590 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:49:57.397590 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:49:57.397590 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:49:57.397590 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:49:57.397590 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:49:57.397590 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:49:57.397590 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:49:57.398438 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:49:57.398438 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:49:57.398438 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:49:57.398438 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:49:57.398438 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:49:57.398438 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:49:57.398438 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:49:57.398438 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:49:57.398438 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:49:57.398438 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:49:57.398438 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 21:49:57.398438 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:49:57.398438 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 79/159] 10.222.0.18 () {56 vars in 1099 bytes} [Sat Aug 6 21:49:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:49:57.405132 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb49a812-bf8f-45df-9926-cd386a7fff0e None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:57.445507 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beac02ac-37fe-418f-9b1e-cc9e4d5a4666 None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:57.445815 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beac02ac-37fe-418f-9b1e-cc9e4d5a4666 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:57.446058 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beac02ac-37fe-418f-9b1e-cc9e4d5a4666 None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:57.447659 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 80/160] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:49:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:49:57.453124 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20048ad0-a29c-4223-94de-3993668da4ad None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:57.453409 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20048ad0-a29c-4223-94de-3993668da4ad None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:57.453796 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20048ad0-a29c-4223-94de-3993668da4ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:57.489567 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-20048ad0-a29c-4223-94de-3993668da4ad None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:49:57.490780 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-20048ad0-a29c-4223-94de-3993668da4ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:49:57.585498 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb49a812-bf8f-45df-9926-cd386a7fff0e None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:57.587516 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:57.598480 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 81/161] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:49:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 21:49:57.638880 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20048ad0-a29c-4223-94de-3993668da4ad None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:57.642038 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e7f820-a7c1-4ecf-b2a0-1a6086ee994b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:57.642942 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e7f820-a7c1-4ecf-b2a0-1a6086ee994b None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:57.643434 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e7f820-a7c1-4ecf-b2a0-1a6086ee994b None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:57.644181 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e7f820-a7c1-4ecf-b2a0-1a6086ee994b None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:57.645321 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e7f820-a7c1-4ecf-b2a0-1a6086ee994b None admin] RBAC: Adding query filter params (name=member) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:49:57.645823 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e7f820-a7c1-4ecf-b2a0-1a6086ee994b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:57.647446 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e7f820-a7c1-4ecf-b2a0-1a6086ee994b None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:57.656627 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 81/162] 10.222.0.18 () {56 vars in 1101 bytes} [Sat Aug 6 21:49:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:49:57.665750 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e7f820-a7c1-4ecf-b2a0-1a6086ee994b None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:57.768073 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:57.787426 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20048ad0-a29c-4223-94de-3993668da4ad None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:57.839518 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78acfeca-2ec7-4b33-9a40-882142e599e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:57.840161 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78acfeca-2ec7-4b33-9a40-882142e599e6 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:57.840456 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78acfeca-2ec7-4b33-9a40-882142e599e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:57.840730 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78acfeca-2ec7-4b33-9a40-882142e599e6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:57.841669 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e7f820-a7c1-4ecf-b2a0-1a6086ee994b None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:57.861361 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78acfeca-2ec7-4b33-9a40-882142e599e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:49:57.866989 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-78acfeca-2ec7-4b33-9a40-882142e599e6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 21:49:57.866989 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:49:57.866989 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:49:57.866989 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:49:57.866989 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:49:57.866989 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:49:57.866989 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:49:57.866989 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:49:57.866989 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:49:57.866989 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:49:57.866989 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:49:57.866989 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:49:57.866989 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:49:57.866989 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:49:57.866989 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:49:57.866989 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:49:57.867877 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:49:57.867877 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:49:57.867877 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:49:57.867877 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:49:57.867877 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:49:57.867877 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:49:57.867877 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:49:57.867877 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:49:57.867877 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:49:57.867877 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:49:57.867877 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:49:57.867877 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:49:57.867877 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:49:57.867877 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:49:57.867877 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:49:57.867877 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:49:57.868695 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:49:57.868695 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:49:57.868695 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:49:57.868695 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:49:57.868695 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:49:57.868695 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:49:57.868695 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:49:57.868695 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:49:57.868695 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:49:57.868695 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:49:57.868695 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 21:49:57.868695 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:49:57.868695 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 82/163] 10.222.0.18 () {56 vars in 1099 bytes} [Sat Aug 6 21:49:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:49:57.874457 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78acfeca-2ec7-4b33-9a40-882142e599e6 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:57.908940 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a7c5a6-d9c1-42a6-adc4-94cc62733b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:57.910466 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a7c5a6-d9c1-42a6-adc4-94cc62733b48 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:57.910961 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a7c5a6-d9c1-42a6-adc4-94cc62733b48 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:57.911535 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a7c5a6-d9c1-42a6-adc4-94cc62733b48 None admin] PATH_INFO: `/v3/users/b7b3753a7de94d39b55d2106b904b48d` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:57.913156 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a7c5a6-d9c1-42a6-adc4-94cc62733b48 None admin] RBAC: Authorizing `identity:get_user(user_id=b7b3753a7de94d39b55d2106b904b48d)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:57.914913 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a7c5a6-d9c1-42a6-adc4-94cc62733b48 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:57.916833 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 82/164] 10.222.0.18 () {56 vars in 1177 bytes} [Sat Aug 6 21:49:57 2022] GET /identity/v3/users/b7b3753a7de94d39b55d2106b904b48d => generated 275 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:49:57.922156 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a7c5a6-d9c1-42a6-adc4-94cc62733b48 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:57.984755 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:58.046375 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78acfeca-2ec7-4b33-9a40-882142e599e6 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:58.100189 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a7c5a6-d9c1-42a6-adc4-94cc62733b48 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:58.152767 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de3ff1a-dfda-44f8-9384-c528a76ad3d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:58.153385 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de3ff1a-dfda-44f8-9384-c528a76ad3d2 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:58.153707 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de3ff1a-dfda-44f8-9384-c528a76ad3d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:58.153988 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de3ff1a-dfda-44f8-9384-c528a76ad3d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:58.177986 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab857379-493e-465c-996b-8e95aac84993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:58.178525 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab857379-493e-465c-996b-8e95aac84993 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:58.178807 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab857379-493e-465c-996b-8e95aac84993 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:58.179081 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab857379-493e-465c-996b-8e95aac84993 None admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:58.179756 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab857379-493e-465c-996b-8e95aac84993 None admin] RBAC: Adding query filter params (name=member) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:49:58.180059 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab857379-493e-465c-996b-8e95aac84993 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:58.181502 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab857379-493e-465c-996b-8e95aac84993 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:58.194825 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 83/165] 10.222.0.18 () {56 vars in 1101 bytes} [Sat Aug 6 21:49:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:49:58.213468 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab857379-493e-465c-996b-8e95aac84993 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:58.231251 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:58.263497 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de3ff1a-dfda-44f8-9384-c528a76ad3d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:58.268091 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de3ff1a-dfda-44f8-9384-c528a76ad3d2 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:58.299142 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:58.405918 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab857379-493e-465c-996b-8e95aac84993 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:58.429677 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 83/166] 10.222.0.18 () {58 vars in 1282 bytes} [Sat Aug 6 21:49:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 06 21:49:58.437405 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de3ff1a-dfda-44f8-9384-c528a76ad3d2 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:58.499020 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d201f0f5-fea4-42f9-99ed-aac759539ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:58.499674 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d201f0f5-fea4-42f9-99ed-aac759539ae0 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:58.499956 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d201f0f5-fea4-42f9-99ed-aac759539ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:58.500232 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d201f0f5-fea4-42f9-99ed-aac759539ae0 None admin] PATH_INFO: `/v3/users/b7b3753a7de94d39b55d2106b904b48d` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:58.502666 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d201f0f5-fea4-42f9-99ed-aac759539ae0 None admin] RBAC: Authorizing `identity:get_user(user_id=b7b3753a7de94d39b55d2106b904b48d)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:58.506680 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d201f0f5-fea4-42f9-99ed-aac759539ae0 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:58.508446 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 84/167] 10.222.0.18 () {56 vars in 1177 bytes} [Sat Aug 6 21:49:58 2022] GET /identity/v3/users/b7b3753a7de94d39b55d2106b904b48d => generated 275 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:49:58.516301 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d201f0f5-fea4-42f9-99ed-aac759539ae0 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:58.625656 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:58.635225 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de3ff1a-dfda-44f8-9384-c528a76ad3d2 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:58.648524 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d201f0f5-fea4-42f9-99ed-aac759539ae0 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:58.685441 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22cda9d-6dbf-4d55-bfba-f5dc29648e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:58.685973 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22cda9d-6dbf-4d55-bfba-f5dc29648e15 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:58.686255 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22cda9d-6dbf-4d55-bfba-f5dc29648e15 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:58.686531 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22cda9d-6dbf-4d55-bfba-f5dc29648e15 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:58.704844 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:58.727832 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07f7eb9-f83f-4f7b-8507-62559be3ca86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:58.728497 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07f7eb9-f83f-4f7b-8507-62559be3ca86 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:58.728824 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07f7eb9-f83f-4f7b-8507-62559be3ca86 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:58.729339 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07f7eb9-f83f-4f7b-8507-62559be3ca86 None admin] PATH_INFO: `/v3/projects/edee8005e8714a9db1c10cfec12857b6` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:58.730676 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07f7eb9-f83f-4f7b-8507-62559be3ca86 None admin] RBAC: Authorizing `identity:get_project(project_id=edee8005e8714a9db1c10cfec12857b6)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:58.732610 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07f7eb9-f83f-4f7b-8507-62559be3ca86 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:58.734443 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 84/168] 10.222.0.18 () {56 vars in 1186 bytes} [Sat Aug 6 21:49:58 2022] GET /identity/v3/projects/edee8005e8714a9db1c10cfec12857b6 => generated 298 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:49:58.735764 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4689aab3-24d3-44b7-83b6-c38b69a65ec7 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:58.739012 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4689aab3-24d3-44b7-83b6-c38b69a65ec7 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:58.739526 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4689aab3-24d3-44b7-83b6-c38b69a65ec7 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:58.741010 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 85/169] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:49:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:49:58.744711 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4689aab3-24d3-44b7-83b6-c38b69a65ec7 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:58.759436 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22cda9d-6dbf-4d55-bfba-f5dc29648e15 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:58.761171 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22cda9d-6dbf-4d55-bfba-f5dc29648e15 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:58.842286 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:58.867619 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 85/170] 10.222.0.18 () {58 vars in 1282 bytes} [Sat Aug 6 21:49:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 21:49:58.869217 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e83f5-d278-4d0a-88b6-2706528c2dd0 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:58.869763 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e83f5-d278-4d0a-88b6-2706528c2dd0 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:58.870299 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e83f5-d278-4d0a-88b6-2706528c2dd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:58.889925 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-6e8e83f5-d278-4d0a-88b6-2706528c2dd0 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:49:58.891577 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-6e8e83f5-d278-4d0a-88b6-2706528c2dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:49:58.951533 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4689aab3-24d3-44b7-83b6-c38b69a65ec7 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:58.969603 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 86/171] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:49:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 21:49:58.970763 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8e83f5-d278-4d0a-88b6-2706528c2dd0 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:59.044110 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0ebe09-0b1a-4118-a2f8-1d8e9bc9404b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:59.045004 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0ebe09-0b1a-4118-a2f8-1d8e9bc9404b None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:59.045419 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0ebe09-0b1a-4118-a2f8-1d8e9bc9404b None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:59.045823 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0ebe09-0b1a-4118-a2f8-1d8e9bc9404b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:59.046765 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0ebe09-0b1a-4118-a2f8-1d8e9bc9404b None admin] RBAC: Adding query filter params (role.id=56e4a96094e9434cbbafb22579c034f5, scope.project.id=edee8005e8714a9db1c10cfec12857b6, user.id=b7b3753a7de94d39b55d2106b904b48d) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:49:59.047260 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0ebe09-0b1a-4118-a2f8-1d8e9bc9404b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:59.048966 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0ebe09-0b1a-4118-a2f8-1d8e9bc9404b None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:59.060944 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 86/172] 10.222.0.18 () {56 vars in 1374 bytes} [Sat Aug 6 21:49:58 2022] GET /identity/v3/role_assignments?user.id=b7b3753a7de94d39b55d2106b904b48d&scope.project.id=edee8005e8714a9db1c10cfec12857b6&role.id=56e4a96094e9434cbbafb22579c034f5 => generated 608 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:49:59.062029 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0ebe09-0b1a-4118-a2f8-1d8e9bc9404b None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:59.083916 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8e83f5-d278-4d0a-88b6-2706528c2dd0 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:59.099752 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:59.132882 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ced301a-14bb-4d4f-8643-e2055e9b35e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:59.133845 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ced301a-14bb-4d4f-8643-e2055e9b35e7 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:59.134259 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ced301a-14bb-4d4f-8643-e2055e9b35e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:59.134663 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ced301a-14bb-4d4f-8643-e2055e9b35e7 None admin] PATH_INFO: `/v3/projects/f89b12d09e6449f990095cb05e6b5627` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:59.144554 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ced301a-14bb-4d4f-8643-e2055e9b35e7 None admin] RBAC: Authorizing `identity:get_project(project_id=f89b12d09e6449f990095cb05e6b5627)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:59.144554 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ced301a-14bb-4d4f-8643-e2055e9b35e7 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:59.144554 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 87/173] 10.222.0.18 () {56 vars in 1186 bytes} [Sat Aug 6 21:49:58 2022] GET /identity/v3/projects/f89b12d09e6449f990095cb05e6b5627 => generated 312 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:49:59.146639 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ced301a-14bb-4d4f-8643-e2055e9b35e7 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:59.171282 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:59.253897 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0ebe09-0b1a-4118-a2f8-1d8e9bc9404b None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:59.314678 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ced301a-14bb-4d4f-8643-e2055e9b35e7 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:59.321789 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd7dc56-f7d5-4b0d-bbd5-983a6c2d3878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:59.322572 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd7dc56-f7d5-4b0d-bbd5-983a6c2d3878 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:59.322946 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd7dc56-f7d5-4b0d-bbd5-983a6c2d3878 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:59.323363 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd7dc56-f7d5-4b0d-bbd5-983a6c2d3878 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:59.336552 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd7dc56-f7d5-4b0d-bbd5-983a6c2d3878 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:49:59.342029 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-acd7dc56-f7d5-4b0d-bbd5-983a6c2d3878 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 21:49:59.342029 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:49:59.342029 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:49:59.342029 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:49:59.342029 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:49:59.342029 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:49:59.342029 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:49:59.342029 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:49:59.342029 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:49:59.342029 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:49:59.342029 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:49:59.342029 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:49:59.342029 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:49:59.342029 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:49:59.342029 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:49:59.342029 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:49:59.343207 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:49:59.343207 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:49:59.343207 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:49:59.343207 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:49:59.343207 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:49:59.343207 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:49:59.343207 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:49:59.343207 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:49:59.343207 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:49:59.343207 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:49:59.343207 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:49:59.343207 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:49:59.343207 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:49:59.343207 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:49:59.343207 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:49:59.343207 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:49:59.344242 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:49:59.344242 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:49:59.344242 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:49:59.344242 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:49:59.344242 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:49:59.344242 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:49:59.344242 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:49:59.344242 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:49:59.344242 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:49:59.344242 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:49:59.344242 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 21:49:59.344242 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:49:59.345850 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 87/174] 10.222.0.18 () {56 vars in 1114 bytes} [Sat Aug 6 21:49:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:49:59.349999 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172ff4de-6f54-4175-b8d5-384a3aa6188a None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:59.349999 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172ff4de-6f54-4175-b8d5-384a3aa6188a None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:59.349999 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172ff4de-6f54-4175-b8d5-384a3aa6188a None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:59.351723 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 88/175] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:49:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:49:59.353025 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7cf71a-1001-497d-bfc6-2289644e0216 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:59.353382 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7cf71a-1001-497d-bfc6-2289644e0216 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:59.353728 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7cf71a-1001-497d-bfc6-2289644e0216 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:59.355383 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 89/176] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:49:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:49:59.356040 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7cf71a-1001-497d-bfc6-2289644e0216 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:59.372643 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf328f28-a114-447f-b80d-68de6cc78a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:59.373662 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf328f28-a114-447f-b80d-68de6cc78a68 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:59.374120 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf328f28-a114-447f-b80d-68de6cc78a68 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:59.374622 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf328f28-a114-447f-b80d-68de6cc78a68 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:59.375533 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf328f28-a114-447f-b80d-68de6cc78a68 None admin] RBAC: Adding query filter params (role.id=56e4a96094e9434cbbafb22579c034f5, scope.project.id=f89b12d09e6449f990095cb05e6b5627, user.id=b7b3753a7de94d39b55d2106b904b48d) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:49:59.376022 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf328f28-a114-447f-b80d-68de6cc78a68 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:59.378754 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf328f28-a114-447f-b80d-68de6cc78a68 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:59.386053 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 88/177] 10.222.0.18 () {56 vars in 1374 bytes} [Sat Aug 6 21:49:59 2022] GET /identity/v3/role_assignments?user.id=b7b3753a7de94d39b55d2106b904b48d&scope.project.id=f89b12d09e6449f990095cb05e6b5627&role.id=56e4a96094e9434cbbafb22579c034f5 => generated 608 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:49:59.387560 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfccf3d-2df6-4cd1-8dcc-00f15f0d260b None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:59.388118 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfccf3d-2df6-4cd1-8dcc-00f15f0d260b None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:59.388614 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfccf3d-2df6-4cd1-8dcc-00f15f0d260b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:59.410169 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-6dfccf3d-2df6-4cd1-8dcc-00f15f0d260b None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:49:59.412441 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-6dfccf3d-2df6-4cd1-8dcc-00f15f0d260b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:49:59.448813 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:59.477422 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7cf71a-1001-497d-bfc6-2289644e0216 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:59.497090 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 89/178] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:49:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 21:49:59.498701 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c5472-56fb-4953-b380-d6f72f8af4cb None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:59.499212 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c5472-56fb-4953-b380-d6f72f8af4cb None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:59.499660 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c5472-56fb-4953-b380-d6f72f8af4cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:59.507996 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:59.520631 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0c1bf3-4ee3-4694-bb69-39ed8578bd34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:59.524446 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0c1bf3-4ee3-4694-bb69-39ed8578bd34 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:59.524446 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0c1bf3-4ee3-4694-bb69-39ed8578bd34 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:59.524446 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0c1bf3-4ee3-4694-bb69-39ed8578bd34 None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:59.524446 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0c1bf3-4ee3-4694-bb69-39ed8578bd34 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:49:59.524446 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0c1bf3-4ee3-4694-bb69-39ed8578bd34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:59.524446 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0c1bf3-4ee3-4694-bb69-39ed8578bd34 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:59.533553 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-fa0c5472-56fb-4953-b380-d6f72f8af4cb None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:49:59.534498 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 90/179] 10.222.0.18 () {56 vars in 1111 bytes} [Sat Aug 6 21:49:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:49:59.536034 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-fa0c5472-56fb-4953-b380-d6f72f8af4cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:49:59.538634 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0c1bf3-4ee3-4694-bb69-39ed8578bd34 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:59.618541 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 90/180] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:49:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 21:49:59.619705 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0c5472-56fb-4953-b380-d6f72f8af4cb None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:59.678256 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0c1bf3-4ee3-4694-bb69-39ed8578bd34 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:59.733614 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0c5472-56fb-4953-b380-d6f72f8af4cb None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:59.736089 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:59.742166 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ee9595-f113-437b-ae0e-64e041eb2bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:59.742904 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee9595-f113-437b-ae0e-64e041eb2bee None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:59.743243 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee9595-f113-437b-ae0e-64e041eb2bee None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:59.743580 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee9595-f113-437b-ae0e-64e041eb2bee None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:59.755578 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ee9595-f113-437b-ae0e-64e041eb2bee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:49:59.762783 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-76ee9595-f113-437b-ae0e-64e041eb2bee None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 21:49:59.762783 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:49:59.762783 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:49:59.762783 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:49:59.762783 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:49:59.762783 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:49:59.762783 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:49:59.762783 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:49:59.762783 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:49:59.762783 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:49:59.762783 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:49:59.762783 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:49:59.762783 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:49:59.762783 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:49:59.762783 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:49:59.762783 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:49:59.763822 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:49:59.763822 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:49:59.763822 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:49:59.763822 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:49:59.763822 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:49:59.763822 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:49:59.763822 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:49:59.763822 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:49:59.763822 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:49:59.763822 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:49:59.763822 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:49:59.763822 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:49:59.763822 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:49:59.763822 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:49:59.763822 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:49:59.763822 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:49:59.764642 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:49:59.764642 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:49:59.764642 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:49:59.764642 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:49:59.764642 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:49:59.764642 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:49:59.764642 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:49:59.764642 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:49:59.764642 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:49:59.764642 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:49:59.764642 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 21:49:59.764642 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:49:59.764642 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 91/181] 10.222.0.18 () {56 vars in 1114 bytes} [Sat Aug 6 21:49:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:49:59.764642 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ee9595-f113-437b-ae0e-64e041eb2bee None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:59.778516 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc47e27-ea6f-4d69-8306-da2392845d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:59.779535 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc47e27-ea6f-4d69-8306-da2392845d22 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:59.780234 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc47e27-ea6f-4d69-8306-da2392845d22 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:59.780690 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc47e27-ea6f-4d69-8306-da2392845d22 None admin] PATH_INFO: `/v3/users/b7b3753a7de94d39b55d2106b904b48d` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:59.782295 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc47e27-ea6f-4d69-8306-da2392845d22 None admin] RBAC: Authorizing `identity:get_user(user_id=b7b3753a7de94d39b55d2106b904b48d)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:59.784139 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc47e27-ea6f-4d69-8306-da2392845d22 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:59.788671 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 91/182] 10.222.0.18 () {56 vars in 1177 bytes} [Sat Aug 6 21:49:59 2022] GET /identity/v3/users/b7b3753a7de94d39b55d2106b904b48d => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:49:59.789776 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc47e27-ea6f-4d69-8306-da2392845d22 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:59.823409 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:49:59.883444 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ee9595-f113-437b-ae0e-64e041eb2bee None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:59.918100 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc47e27-ea6f-4d69-8306-da2392845d22 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:49:59.919743 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfa0404-5898-407a-b729-4844bcde5f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:59.920352 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfa0404-5898-407a-b729-4844bcde5f7c None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:59.920696 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfa0404-5898-407a-b729-4844bcde5f7c None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:59.920965 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfa0404-5898-407a-b729-4844bcde5f7c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:59.933318 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfa0404-5898-407a-b729-4844bcde5f7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:49:59.938859 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-dbfa0404-5898-407a-b729-4844bcde5f7c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 21:49:59.938859 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:49:59.938859 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:49:59.938859 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:49:59.938859 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:49:59.938859 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:49:59.938859 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:49:59.938859 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:49:59.938859 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:49:59.938859 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:49:59.938859 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:49:59.938859 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:49:59.938859 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:49:59.938859 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:49:59.938859 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:49:59.938859 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:49:59.939985 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:49:59.939985 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:49:59.939985 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:49:59.939985 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:49:59.939985 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:49:59.939985 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:49:59.939985 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:49:59.939985 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:49:59.939985 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:49:59.939985 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:49:59.939985 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:49:59.939985 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:49:59.939985 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:49:59.939985 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:49:59.939985 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:49:59.939985 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:49:59.941358 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:49:59.941358 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:49:59.941358 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:49:59.941358 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:49:59.941358 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:49:59.941358 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:49:59.941358 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:49:59.941358 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:49:59.941358 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:49:59.941358 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:49:59.941358 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 21:49:59.941358 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:49:59.941358 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 92/183] 10.222.0.18 () {56 vars in 1096 bytes} [Sat Aug 6 21:49:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:49:59.941358 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfa0404-5898-407a-b729-4844bcde5f7c None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:49:59.960810 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a0831b-7ca9-4614-95f0-820f08ea1e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:49:59.961711 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a0831b-7ca9-4614-95f0-820f08ea1e97 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:49:59.962344 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a0831b-7ca9-4614-95f0-820f08ea1e97 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:49:59.962783 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a0831b-7ca9-4614-95f0-820f08ea1e97 None admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:49:59.963706 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a0831b-7ca9-4614-95f0-820f08ea1e97 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:49:59.964164 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a0831b-7ca9-4614-95f0-820f08ea1e97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:49:59.965780 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a0831b-7ca9-4614-95f0-820f08ea1e97 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:49:59.977688 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 92/184] 10.222.0.18 () {56 vars in 1111 bytes} [Sat Aug 6 21:49:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:49:59.978829 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a0831b-7ca9-4614-95f0-820f08ea1e97 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:00.032863 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:00.053850 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfa0404-5898-407a-b729-4844bcde5f7c None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:00.062485 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:00.095478 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa38c2e5-03dd-44f3-bd10-0bab52ffb7a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:00.096235 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa38c2e5-03dd-44f3-bd10-0bab52ffb7a7 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:00.096525 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa38c2e5-03dd-44f3-bd10-0bab52ffb7a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:00.097776 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa38c2e5-03dd-44f3-bd10-0bab52ffb7a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:00.100394 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a0831b-7ca9-4614-95f0-820f08ea1e97 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:00.138356 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb681667-e9a1-4a27-bedc-c11ed91e9917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:00.138957 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb681667-e9a1-4a27-bedc-c11ed91e9917 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:00.139233 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb681667-e9a1-4a27-bedc-c11ed91e9917 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:00.139499 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb681667-e9a1-4a27-bedc-c11ed91e9917 None admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:00.140159 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb681667-e9a1-4a27-bedc-c11ed91e9917 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:00.140485 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb681667-e9a1-4a27-bedc-c11ed91e9917 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:00.141855 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb681667-e9a1-4a27-bedc-c11ed91e9917 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:00.150401 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 93/185] 10.222.0.18 () {56 vars in 1099 bytes} [Sat Aug 6 21:49:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:00.151202 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb681667-e9a1-4a27-bedc-c11ed91e9917 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:00.177977 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa38c2e5-03dd-44f3-bd10-0bab52ffb7a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:00.179708 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa38c2e5-03dd-44f3-bd10-0bab52ffb7a7 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:00.253185 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:00.263401 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb681667-e9a1-4a27-bedc-c11ed91e9917 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:00.297634 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 93/186] 10.222.0.18 () {58 vars in 1282 bytes} [Sat Aug 6 21:49:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 21:50:00.297634 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bba024-865b-4577-a189-f8458920baea None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:00.297634 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bba024-865b-4577-a189-f8458920baea None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:00.297634 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bba024-865b-4577-a189-f8458920baea None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:00.297634 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 94/187] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:50:00 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:50:00.297634 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bba024-865b-4577-a189-f8458920baea None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:00.306498 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:00.306647 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560374bd-59f5-4c29-9b49-50c6587818a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:00.307240 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560374bd-59f5-4c29-9b49-50c6587818a2 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:00.307525 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560374bd-59f5-4c29-9b49-50c6587818a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:00.307799 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560374bd-59f5-4c29-9b49-50c6587818a2 None admin] PATH_INFO: `/v3/users/57cdf1d257184e108edfe80e84eb29db` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:00.322554 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560374bd-59f5-4c29-9b49-50c6587818a2 None admin] RBAC: Authorizing `identity:get_user(user_id=57cdf1d257184e108edfe80e84eb29db)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:00.324018 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560374bd-59f5-4c29-9b49-50c6587818a2 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:00.325695 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 94/188] 10.222.0.18 () {56 vars in 1177 bytes} [Sat Aug 6 21:50:00 2022] GET /identity/v3/users/57cdf1d257184e108edfe80e84eb29db => generated 283 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:00.326466 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560374bd-59f5-4c29-9b49-50c6587818a2 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:00.404095 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bba024-865b-4577-a189-f8458920baea None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:00.430986 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560374bd-59f5-4c29-9b49-50c6587818a2 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:00.437862 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7dd6ac-7031-4d38-b6b6-e167d527a782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:00.438485 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7dd6ac-7031-4d38-b6b6-e167d527a782 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:00.438757 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7dd6ac-7031-4d38-b6b6-e167d527a782 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:00.439021 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7dd6ac-7031-4d38-b6b6-e167d527a782 None admin] PATH_INFO: `/v3/users/57cdf1d257184e108edfe80e84eb29db` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:00.440211 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7dd6ac-7031-4d38-b6b6-e167d527a782 None admin] RBAC: Authorizing `identity:get_user(user_id=57cdf1d257184e108edfe80e84eb29db)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:00.442127 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7dd6ac-7031-4d38-b6b6-e167d527a782 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:00.443723 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 95/189] 10.222.0.18 () {56 vars in 1177 bytes} [Sat Aug 6 21:50:00 2022] GET /identity/v3/users/57cdf1d257184e108edfe80e84eb29db => generated 283 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:00.444869 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517bb65-091d-4989-80eb-4d7e36698dce None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:00.445136 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517bb65-091d-4989-80eb-4d7e36698dce None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:00.445412 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517bb65-091d-4989-80eb-4d7e36698dce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:00.463423 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-9517bb65-091d-4989-80eb-4d7e36698dce None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:50:00.464638 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-9517bb65-091d-4989-80eb-4d7e36698dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:50:00.465953 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daef08ee-09b5-4410-bc4d-aeb9ab949d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:00.466670 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef08ee-09b5-4410-bc4d-aeb9ab949d8f None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:00.466951 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef08ee-09b5-4410-bc4d-aeb9ab949d8f None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:00.467226 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef08ee-09b5-4410-bc4d-aeb9ab949d8f None admin] PATH_INFO: `/v3/projects/edee8005e8714a9db1c10cfec12857b6` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:00.468346 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daef08ee-09b5-4410-bc4d-aeb9ab949d8f None admin] RBAC: Authorizing `identity:get_project(project_id=edee8005e8714a9db1c10cfec12857b6)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:00.469806 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daef08ee-09b5-4410-bc4d-aeb9ab949d8f None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:00.471673 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 95/190] 10.222.0.18 () {56 vars in 1186 bytes} [Sat Aug 6 21:50:00 2022] GET /identity/v3/projects/edee8005e8714a9db1c10cfec12857b6 => generated 298 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:00.473348 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e5fc9d-5775-45d9-b090-8f86b99e6fc3 None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:00.473630 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e5fc9d-5775-45d9-b090-8f86b99e6fc3 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:00.473903 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e5fc9d-5775-45d9-b090-8f86b99e6fc3 None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:00.475111 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 96/191] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:50:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:50:00.476672 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e5fc9d-5775-45d9-b090-8f86b99e6fc3 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:00.494914 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:00.538621 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 96/192] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:50:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 21:50:00.541333 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9517bb65-091d-4989-80eb-4d7e36698dce None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:00.544330 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:00.582258 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e5fc9d-5775-45d9-b090-8f86b99e6fc3 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:00.614596 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f1afc8-3a22-4c48-8da2-d6205e8f4d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:00.615195 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f1afc8-3a22-4c48-8da2-d6205e8f4d4e None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:00.615475 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f1afc8-3a22-4c48-8da2-d6205e8f4d4e None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:00.615742 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f1afc8-3a22-4c48-8da2-d6205e8f4d4e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:00.644545 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9517bb65-091d-4989-80eb-4d7e36698dce None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:00.677366 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e8e34d-ac52-4eb4-9b28-08d6afa4cfa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:00.677973 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e8e34d-ac52-4eb4-9b28-08d6afa4cfa0 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:00.678248 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e8e34d-ac52-4eb4-9b28-08d6afa4cfa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:00.678515 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e8e34d-ac52-4eb4-9b28-08d6afa4cfa0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:00.680039 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f1afc8-3a22-4c48-8da2-d6205e8f4d4e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:00.681600 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f1afc8-3a22-4c48-8da2-d6205e8f4d4e None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:00.698685 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:00.744784 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e8e34d-ac52-4eb4-9b28-08d6afa4cfa0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:00.746266 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e8e34d-ac52-4eb4-9b28-08d6afa4cfa0 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:00.748856 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:00.781762 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 97/193] 10.222.0.18 () {58 vars in 1282 bytes} [Sat Aug 6 21:50:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 21:50:00.781762 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f1afc8-3a22-4c48-8da2-d6205e8f4d4e None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:00.847312 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 97/194] 10.222.0.18 () {58 vars in 1282 bytes} [Sat Aug 6 21:50:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 21:50:00.848613 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3222d42-bb0a-4df0-baf8-245bc5c026f0 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:00.848900 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3222d42-bb0a-4df0-baf8-245bc5c026f0 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:00.849174 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3222d42-bb0a-4df0-baf8-245bc5c026f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:00.866506 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-c3222d42-bb0a-4df0-baf8-245bc5c026f0 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:50:00.867692 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-c3222d42-bb0a-4df0-baf8-245bc5c026f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:50:00.884359 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f1afc8-3a22-4c48-8da2-d6205e8f4d4e None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:00.900938 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:00.918184 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddf3c66-9a73-4738-99c9-2a70f5ae8557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:00.918781 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddf3c66-9a73-4738-99c9-2a70f5ae8557 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:00.919063 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddf3c66-9a73-4738-99c9-2a70f5ae8557 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:00.919341 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddf3c66-9a73-4738-99c9-2a70f5ae8557 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:00.920044 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddf3c66-9a73-4738-99c9-2a70f5ae8557 None admin] RBAC: Adding query filter params (role.id=93c058ac49e242fd8517c80c552dea2d, scope.project.id=edee8005e8714a9db1c10cfec12857b6, user.id=b7b3753a7de94d39b55d2106b904b48d) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:00.920346 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddf3c66-9a73-4738-99c9-2a70f5ae8557 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:00.921820 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddf3c66-9a73-4738-99c9-2a70f5ae8557 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:00.928013 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 98/195] 10.222.0.18 () {56 vars in 1374 bytes} [Sat Aug 6 21:50:00 2022] GET /identity/v3/role_assignments?user.id=b7b3753a7de94d39b55d2106b904b48d&scope.project.id=edee8005e8714a9db1c10cfec12857b6&role.id=93c058ac49e242fd8517c80c552dea2d => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:00.928825 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddf3c66-9a73-4738-99c9-2a70f5ae8557 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:00.948309 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 98/196] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:50:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 21:50:00.949079 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3222d42-bb0a-4df0-baf8-245bc5c026f0 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:00.990752 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:01.053489 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddf3c66-9a73-4738-99c9-2a70f5ae8557 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:01.066778 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3222d42-bb0a-4df0-baf8-245bc5c026f0 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:01.093717 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0c6efd-230c-4d82-bd7f-0119a7792018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:01.094421 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0c6efd-230c-4d82-bd7f-0119a7792018 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:01.094612 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0c6efd-230c-4d82-bd7f-0119a7792018 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:01.094880 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0c6efd-230c-4d82-bd7f-0119a7792018 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:01.095989 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0c6efd-230c-4d82-bd7f-0119a7792018 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:01.097723 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0c6efd-230c-4d82-bd7f-0119a7792018 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:01.099256 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 99/197] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:50:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:01.100458 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0c6efd-230c-4d82-bd7f-0119a7792018 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:01.108802 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c651b51-1381-40de-92d0-1d44a8f46aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:01.109697 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c651b51-1381-40de-92d0-1d44a8f46aea None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:01.109697 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c651b51-1381-40de-92d0-1d44a8f46aea None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:01.109973 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c651b51-1381-40de-92d0-1d44a8f46aea None admin] PATH_INFO: `/v3/projects/0bfd7af746994807b3d8de6f7292ef02` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:01.111162 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c651b51-1381-40de-92d0-1d44a8f46aea None admin] RBAC: Authorizing `identity:get_project(project_id=0bfd7af746994807b3d8de6f7292ef02)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:01.114001 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c651b51-1381-40de-92d0-1d44a8f46aea None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:01.115017 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 99/198] 10.222.0.18 () {56 vars in 1186 bytes} [Sat Aug 6 21:50:00 2022] GET /identity/v3/projects/0bfd7af746994807b3d8de6f7292ef02 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:01.115627 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c651b51-1381-40de-92d0-1d44a8f46aea None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:01.160162 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:01.212186 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0c6efd-230c-4d82-bd7f-0119a7792018 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:01.228245 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c651b51-1381-40de-92d0-1d44a8f46aea None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:01.229750 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:01.246023 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b03a7c0-1142-4f04-b629-1c837c72efd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:01.246722 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b03a7c0-1142-4f04-b629-1c837c72efd1 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:01.247061 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b03a7c0-1142-4f04-b629-1c837c72efd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:01.247396 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b03a7c0-1142-4f04-b629-1c837c72efd1 None admin] PATH_INFO: `/v3/projects/0bfd7af746994807b3d8de6f7292ef02` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:01.248634 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b03a7c0-1142-4f04-b629-1c837c72efd1 None admin] RBAC: Authorizing `identity:get_project(project_id=0bfd7af746994807b3d8de6f7292ef02)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:01.250078 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b03a7c0-1142-4f04-b629-1c837c72efd1 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:01.251689 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 100/199] 10.222.0.18 () {56 vars in 1186 bytes} [Sat Aug 6 21:50:01 2022] GET /identity/v3/projects/0bfd7af746994807b3d8de6f7292ef02 => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:01.252435 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b03a7c0-1142-4f04-b629-1c837c72efd1 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:01.266004 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942a2c46-cbbd-483d-82e2-bde1ac0c49d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:01.266669 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942a2c46-cbbd-483d-82e2-bde1ac0c49d1 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:01.266949 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942a2c46-cbbd-483d-82e2-bde1ac0c49d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:01.267217 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942a2c46-cbbd-483d-82e2-bde1ac0c49d1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:01.277522 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942a2c46-cbbd-483d-82e2-bde1ac0c49d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:50:01.282222 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-942a2c46-cbbd-483d-82e2-bde1ac0c49d1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 21:50:01.282222 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:01.282222 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:01.282222 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:01.282222 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:01.282222 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:01.282222 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:01.282222 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:01.282222 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:01.282222 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:01.282222 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:01.282222 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:01.282222 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:50:01.282222 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:50:01.282222 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:50:01.282222 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:50:01.283246 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:50:01.283246 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:50:01.283246 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:50:01.283246 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:50:01.283246 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:01.283246 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:01.283246 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:01.283246 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:01.283246 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:01.283246 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:01.283246 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:01.283246 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:01.283246 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:01.283246 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:01.283246 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:01.283246 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:01.289385 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:01.289385 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:01.289385 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:01.289385 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:01.289385 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:50:01.289385 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:50:01.289385 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:50:01.289385 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:50:01.289385 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:50:01.289385 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:50:01.289385 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 21:50:01.289385 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:50:01.289385 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 100/200] 10.222.0.18 () {56 vars in 1099 bytes} [Sat Aug 6 21:50:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:01.289385 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942a2c46-cbbd-483d-82e2-bde1ac0c49d1 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:01.351699 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b03a7c0-1142-4f04-b629-1c837c72efd1 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:01.369190 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:01.384522 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942a2c46-cbbd-483d-82e2-bde1ac0c49d1 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:01.387025 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83a734f-57bd-42fd-8905-c1ac6ee05a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:01.387627 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83a734f-57bd-42fd-8905-c1ac6ee05a7c None admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:01.387902 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83a734f-57bd-42fd-8905-c1ac6ee05a7c None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:01.388172 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83a734f-57bd-42fd-8905-c1ac6ee05a7c None admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:01.388843 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83a734f-57bd-42fd-8905-c1ac6ee05a7c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:01.390271 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83a734f-57bd-42fd-8905-c1ac6ee05a7c None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:01.417787 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e96aab3-2ee2-443e-acee-0304a3d3ff61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:01.418385 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e96aab3-2ee2-443e-acee-0304a3d3ff61 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:01.418663 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e96aab3-2ee2-443e-acee-0304a3d3ff61 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:01.418935 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e96aab3-2ee2-443e-acee-0304a3d3ff61 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:01.419655 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e96aab3-2ee2-443e-acee-0304a3d3ff61 None admin] RBAC: Adding query filter params (role.id=93c058ac49e242fd8517c80c552dea2d, scope.project.id=0bfd7af746994807b3d8de6f7292ef02, user.id=57cdf1d257184e108edfe80e84eb29db) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:01.419961 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e96aab3-2ee2-443e-acee-0304a3d3ff61 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:01.421417 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e96aab3-2ee2-443e-acee-0304a3d3ff61 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:01.427345 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 101/201] 10.222.0.18 () {56 vars in 1374 bytes} [Sat Aug 6 21:50:01 2022] GET /identity/v3/role_assignments?user.id=57cdf1d257184e108edfe80e84eb29db&scope.project.id=0bfd7af746994807b3d8de6f7292ef02&role.id=93c058ac49e242fd8517c80c552dea2d => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:01.428113 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e96aab3-2ee2-443e-acee-0304a3d3ff61 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:01.457645 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:01.466537 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 101/202] 10.222.0.18 () {60 vars in 1132 bytes} [Sat Aug 6 21:50:01 2022] POST /identity/v3/users => generated 297 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 21:50:01.468765 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83a734f-57bd-42fd-8905-c1ac6ee05a7c None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:01.557369 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e96aab3-2ee2-443e-acee-0304a3d3ff61 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:01.607601 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978016e5-e05b-4cd3-ab50-eb21626eadb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:01.608691 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978016e5-e05b-4cd3-ab50-eb21626eadb8 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:01.609201 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978016e5-e05b-4cd3-ab50-eb21626eadb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:01.609684 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978016e5-e05b-4cd3-ab50-eb21626eadb8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:01.610691 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978016e5-e05b-4cd3-ab50-eb21626eadb8 None admin] RBAC: Adding query filter params (role.id=4b8dcf0491fd4072acc69fd894bf77d0, scope.project.id=0bfd7af746994807b3d8de6f7292ef02, user.id=57cdf1d257184e108edfe80e84eb29db) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:01.611113 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978016e5-e05b-4cd3-ab50-eb21626eadb8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:01.612848 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978016e5-e05b-4cd3-ab50-eb21626eadb8 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:01.621482 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 102/203] 10.222.0.18 () {56 vars in 1374 bytes} [Sat Aug 6 21:50:01 2022] GET /identity/v3/role_assignments?user.id=57cdf1d257184e108edfe80e84eb29db&scope.project.id=0bfd7af746994807b3d8de6f7292ef02&role.id=4b8dcf0491fd4072acc69fd894bf77d0 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:01.628336 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83a734f-57bd-42fd-8905-c1ac6ee05a7c None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:01.673242 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b0c0a9-ed88-44a0-b998-71eb4516847b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:01.673935 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b0c0a9-ed88-44a0-b998-71eb4516847b None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:01.674222 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b0c0a9-ed88-44a0-b998-71eb4516847b None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:01.674549 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b0c0a9-ed88-44a0-b998-71eb4516847b None admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:01.675455 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b0c0a9-ed88-44a0-b998-71eb4516847b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:01.675678 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b0c0a9-ed88-44a0-b998-71eb4516847b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:01.677173 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b0c0a9-ed88-44a0-b998-71eb4516847b None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:01.687606 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 102/204] 10.222.0.18 () {56 vars in 1101 bytes} [Sat Aug 6 21:50:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:01.694957 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978016e5-e05b-4cd3-ab50-eb21626eadb8 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:01.802473 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:01.812331 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978016e5-e05b-4cd3-ab50-eb21626eadb8 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:01.850955 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f39bfe-54dc-48ab-aad5-383ed916ce76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:01.851968 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f39bfe-54dc-48ab-aad5-383ed916ce76 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:01.852465 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f39bfe-54dc-48ab-aad5-383ed916ce76 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:01.852929 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f39bfe-54dc-48ab-aad5-383ed916ce76 None admin] PATH_INFO: `/v3/users/7e19a292a7d749c291e942bc238399ba` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:01.854262 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f39bfe-54dc-48ab-aad5-383ed916ce76 None admin] RBAC: Authorizing `identity:get_user(user_id=7e19a292a7d749c291e942bc238399ba)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:01.855762 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f39bfe-54dc-48ab-aad5-383ed916ce76 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:01.857944 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 103/205] 10.222.0.18 () {56 vars in 1177 bytes} [Sat Aug 6 21:50:01 2022] GET /identity/v3/users/7e19a292a7d749c291e942bc238399ba => generated 289 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:01.864079 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b0c0a9-ed88-44a0-b998-71eb4516847b None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:01.901366 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:01.978330 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b0c0a9-ed88-44a0-b998-71eb4516847b None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:02.014078 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a952ba1-b478-45b7-a688-65d511fadefe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:02.014745 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a952ba1-b478-45b7-a688-65d511fadefe None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:02.015024 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a952ba1-b478-45b7-a688-65d511fadefe None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:02.015293 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a952ba1-b478-45b7-a688-65d511fadefe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:02.083869 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a952ba1-b478-45b7-a688-65d511fadefe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:02.085539 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a952ba1-b478-45b7-a688-65d511fadefe None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:02.119811 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:02.204632 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 103/206] 10.222.0.18 () {58 vars in 1282 bytes} [Sat Aug 6 21:50:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 06 21:50:02.210325 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f39bfe-54dc-48ab-aad5-383ed916ce76 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:02.348855 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f39bfe-54dc-48ab-aad5-383ed916ce76 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:02.395022 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0428975-2eec-48da-bdf9-46308d1a54f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:02.395586 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0428975-2eec-48da-bdf9-46308d1a54f4 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:02.395862 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0428975-2eec-48da-bdf9-46308d1a54f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:02.396132 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0428975-2eec-48da-bdf9-46308d1a54f4 None admin] PATH_INFO: `/v3/projects/edee8005e8714a9db1c10cfec12857b6` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:02.397345 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0428975-2eec-48da-bdf9-46308d1a54f4 None admin] RBAC: Authorizing `identity:get_project(project_id=edee8005e8714a9db1c10cfec12857b6)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:02.398931 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0428975-2eec-48da-bdf9-46308d1a54f4 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:02.400577 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 104/207] 10.222.0.18 () {56 vars in 1186 bytes} [Sat Aug 6 21:50:02 2022] GET /identity/v3/projects/edee8005e8714a9db1c10cfec12857b6 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:02.406408 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a952ba1-b478-45b7-a688-65d511fadefe None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:02.536876 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a952ba1-b478-45b7-a688-65d511fadefe None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:02.579800 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a50f2f-5a8e-48ea-be40-bac976448993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:02.580426 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a50f2f-5a8e-48ea-be40-bac976448993 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:02.580711 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a50f2f-5a8e-48ea-be40-bac976448993 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:02.581007 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a50f2f-5a8e-48ea-be40-bac976448993 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:02.581796 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a50f2f-5a8e-48ea-be40-bac976448993 None admin] RBAC: Adding query filter params (role.id=817eabff1ca5470993116be35777000a, scope.project.id=edee8005e8714a9db1c10cfec12857b6, user.id=7e19a292a7d749c291e942bc238399ba) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:02.582105 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a50f2f-5a8e-48ea-be40-bac976448993 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:02.583538 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a50f2f-5a8e-48ea-be40-bac976448993 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:02.585355 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:02.592696 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 104/208] 10.222.0.18 () {56 vars in 1374 bytes} [Sat Aug 6 21:50:02 2022] GET /identity/v3/role_assignments?user.id=7e19a292a7d749c291e942bc238399ba&scope.project.id=edee8005e8714a9db1c10cfec12857b6&role.id=817eabff1ca5470993116be35777000a => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:03.664031 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6a1944-7bbc-47a0-94da-d31aafe368a1 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:03.664605 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6a1944-7bbc-47a0-94da-d31aafe368a1 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:03.664605 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6a1944-7bbc-47a0-94da-d31aafe368a1 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:03.665872 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 105/209] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:50:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:50:03.671118 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f7f00e-600e-4677-913b-d8845bf679c0 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:03.671509 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f7f00e-600e-4677-913b-d8845bf679c0 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:03.671668 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f7f00e-600e-4677-913b-d8845bf679c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:03.690624 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-47f7f00e-600e-4677-913b-d8845bf679c0 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:50:03.691904 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-47f7f00e-600e-4677-913b-d8845bf679c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:50:03.761252 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 105/210] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:50:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 21:50:03.783217 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6a1944-7bbc-47a0-94da-d31aafe368a1 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:03.812777 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:03.865818 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0d50d4-aa02-46c8-92a5-faee1ccac071 None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:03.866435 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0d50d4-aa02-46c8-92a5-faee1ccac071 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:03.866925 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0d50d4-aa02-46c8-92a5-faee1ccac071 None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:03.868478 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 106/211] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:50:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 21:50:03.873718 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554fc2d3-a68c-4a92-9976-af77b0983506 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:03.873995 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554fc2d3-a68c-4a92-9976-af77b0983506 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:03.874264 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554fc2d3-a68c-4a92-9976-af77b0983506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:03.892490 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6a1944-7bbc-47a0-94da-d31aafe368a1 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:03.893232 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-554fc2d3-a68c-4a92-9976-af77b0983506 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:50:03.894862 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-554fc2d3-a68c-4a92-9976-af77b0983506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:50:03.930198 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ef5a4c-3aec-4748-833b-48f7079a1ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:03.931242 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ef5a4c-3aec-4748-833b-48f7079a1ed8 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:03.931815 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ef5a4c-3aec-4748-833b-48f7079a1ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:03.932274 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ef5a4c-3aec-4748-833b-48f7079a1ed8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:03.944432 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ef5a4c-3aec-4748-833b-48f7079a1ed8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:50:03.950055 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-a0ef5a4c-3aec-4748-833b-48f7079a1ed8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 21:50:03.950055 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:03.950055 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:03.950055 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:03.950055 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:03.950055 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:03.950055 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:03.950055 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:03.950055 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:03.950055 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:03.950055 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:03.950055 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:03.950055 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:50:03.950055 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:50:03.950055 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:50:03.950055 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:50:03.950912 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:50:03.950912 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:50:03.950912 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:50:03.950912 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:50:03.950912 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:03.950912 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:03.950912 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:03.950912 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:03.950912 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:03.950912 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:03.950912 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:03.950912 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:03.950912 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:03.950912 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:03.950912 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:03.950912 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:03.951703 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:03.951703 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:03.951703 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:03.951703 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:03.951703 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:50:03.951703 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:50:03.951703 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:50:03.951703 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:50:03.951703 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:50:03.951703 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:50:03.951703 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 21:50:03.951703 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:50:03.954211 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 106/212] 10.222.0.18 () {56 vars in 1114 bytes} [Sat Aug 6 21:50:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:03.961418 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ef5a4c-3aec-4748-833b-48f7079a1ed8 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:03.968164 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 107/213] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:50:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 21:50:03.991499 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554fc2d3-a68c-4a92-9976-af77b0983506 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:04.055964 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:04.057799 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:04.075783 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ef5a4c-3aec-4748-833b-48f7079a1ed8 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:04.102002 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554fc2d3-a68c-4a92-9976-af77b0983506 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:04.113537 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e000cd-eb0d-4a26-ac2b-dd8fd5474f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:04.114304 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e000cd-eb0d-4a26-ac2b-dd8fd5474f2e None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:04.114771 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e000cd-eb0d-4a26-ac2b-dd8fd5474f2e None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:04.115041 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e000cd-eb0d-4a26-ac2b-dd8fd5474f2e None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:04.115949 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e000cd-eb0d-4a26-ac2b-dd8fd5474f2e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:04.116328 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e000cd-eb0d-4a26-ac2b-dd8fd5474f2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:04.118212 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e000cd-eb0d-4a26-ac2b-dd8fd5474f2e None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:04.126871 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 107/214] 10.222.0.18 () {56 vars in 1111 bytes} [Sat Aug 6 21:50:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:04.132942 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073ea095-055a-4f5e-be76-dc61266f0fc7 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:04.133223 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073ea095-055a-4f5e-be76-dc61266f0fc7 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:04.133496 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073ea095-055a-4f5e-be76-dc61266f0fc7 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:04.134891 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 108/215] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:50:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:50:04.136115 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073ea095-055a-4f5e-be76-dc61266f0fc7 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:04.140521 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a47ff5-ba2c-48af-af04-d74f96a6e9a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:04.140521 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a47ff5-ba2c-48af-af04-d74f96a6e9a0 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:04.140751 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a47ff5-ba2c-48af-af04-d74f96a6e9a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:04.140968 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a47ff5-ba2c-48af-af04-d74f96a6e9a0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:04.154208 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a47ff5-ba2c-48af-af04-d74f96a6e9a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:50:04.159561 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-67a47ff5-ba2c-48af-af04-d74f96a6e9a0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 21:50:04.159561 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:04.159561 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:04.159561 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:04.159561 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:04.159561 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:04.159561 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:04.159561 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:04.159561 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:04.159561 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:04.159561 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:04.159561 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:04.159561 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:50:04.159561 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:50:04.159561 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:50:04.159561 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:50:04.160445 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:50:04.160445 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:50:04.160445 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:50:04.160445 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:50:04.160445 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:04.160445 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:04.160445 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:04.160445 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:04.160445 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:04.160445 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:04.160445 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:04.160445 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:04.160445 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:04.160445 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:04.160445 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:04.160445 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:04.161257 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:04.161257 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:04.161257 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:04.161257 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:04.161257 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:50:04.161257 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:50:04.161257 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:50:04.161257 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:50:04.161257 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:50:04.161257 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:50:04.161257 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 21:50:04.161257 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:50:04.162506 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 108/216] 10.222.0.18 () {56 vars in 1096 bytes} [Sat Aug 6 21:50:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 21:50:04.163573 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d624b91b-ecea-4356-8f33-9163e48d04ad None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:04.163853 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d624b91b-ecea-4356-8f33-9163e48d04ad None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:04.164125 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d624b91b-ecea-4356-8f33-9163e48d04ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:04.185487 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-d624b91b-ecea-4356-8f33-9163e48d04ad None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:50:04.186933 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-d624b91b-ecea-4356-8f33-9163e48d04ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:50:04.244096 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073ea095-055a-4f5e-be76-dc61266f0fc7 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:04.252276 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 109/217] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:50:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 21:50:04.254297 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d624b91b-ecea-4356-8f33-9163e48d04ad None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:04.288715 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731082ad-58f8-43b0-b6c5-ccfe2c1c5cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:04.289349 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731082ad-58f8-43b0-b6c5-ccfe2c1c5cff None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:04.289651 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731082ad-58f8-43b0-b6c5-ccfe2c1c5cff None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:04.289924 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731082ad-58f8-43b0-b6c5-ccfe2c1c5cff None admin] PATH_INFO: `/v3/users/57cdf1d257184e108edfe80e84eb29db` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:04.291033 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731082ad-58f8-43b0-b6c5-ccfe2c1c5cff None admin] RBAC: Authorizing `identity:get_user(user_id=57cdf1d257184e108edfe80e84eb29db)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:04.292505 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731082ad-58f8-43b0-b6c5-ccfe2c1c5cff None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:04.298747 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 109/218] 10.222.0.18 () {56 vars in 1177 bytes} [Sat Aug 6 21:50:04 2022] GET /identity/v3/users/57cdf1d257184e108edfe80e84eb29db => generated 283 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:04.298747 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f79f85-eedb-4bfb-bd3f-ece351409667 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:04.298747 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f79f85-eedb-4bfb-bd3f-ece351409667 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:04.298747 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f79f85-eedb-4bfb-bd3f-ece351409667 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:04.298747 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 110/219] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:50:04 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:50:04.298747 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f79f85-eedb-4bfb-bd3f-ece351409667 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:04.312095 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:04.316641 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:04.363942 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d624b91b-ecea-4356-8f33-9163e48d04ad None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:04.396750 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034b24d2-9e09-4839-9dcb-aed97402741e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:04.397369 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b24d2-9e09-4839-9dcb-aed97402741e None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:04.397647 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b24d2-9e09-4839-9dcb-aed97402741e None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:04.397911 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b24d2-9e09-4839-9dcb-aed97402741e None admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:04.398572 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034b24d2-9e09-4839-9dcb-aed97402741e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:04.398868 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034b24d2-9e09-4839-9dcb-aed97402741e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:04.400255 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034b24d2-9e09-4839-9dcb-aed97402741e None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:04.401839 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f79f85-eedb-4bfb-bd3f-ece351409667 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:04.408029 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 110/220] 10.222.0.18 () {56 vars in 1099 bytes} [Sat Aug 6 21:50:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:04.413384 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034b24d2-9e09-4839-9dcb-aed97402741e None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:04.436579 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb35e6f1-d8bb-4cbf-8297-09fffdee2a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:04.437139 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb35e6f1-d8bb-4cbf-8297-09fffdee2a6a None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:04.437415 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb35e6f1-d8bb-4cbf-8297-09fffdee2a6a None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:04.437688 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb35e6f1-d8bb-4cbf-8297-09fffdee2a6a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:04.438772 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb35e6f1-d8bb-4cbf-8297-09fffdee2a6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:04.440225 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb35e6f1-d8bb-4cbf-8297-09fffdee2a6a None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:04.441797 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 111/221] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:50:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:04.442880 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cefd4a-1a88-4cb1-b5c9-20b650f7106d None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:04.443157 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cefd4a-1a88-4cb1-b5c9-20b650f7106d None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:04.443427 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cefd4a-1a88-4cb1-b5c9-20b650f7106d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:04.461249 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-83cefd4a-1a88-4cb1-b5c9-20b650f7106d None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:50:04.462425 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-83cefd4a-1a88-4cb1-b5c9-20b650f7106d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:50:04.514296 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034b24d2-9e09-4839-9dcb-aed97402741e None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:04.524087 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:04.527314 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 112/222] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:50:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 21:50:04.528172 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cefd4a-1a88-4cb1-b5c9-20b650f7106d None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:04.542959 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:04.548451 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45d3360-36c9-46e4-be74-d72ef58c151e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:04.549152 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45d3360-36c9-46e4-be74-d72ef58c151e None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:04.549428 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45d3360-36c9-46e4-be74-d72ef58c151e None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:04.549926 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45d3360-36c9-46e4-be74-d72ef58c151e None admin] PATH_INFO: `/v3/projects/0bfd7af746994807b3d8de6f7292ef02` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:04.551034 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45d3360-36c9-46e4-be74-d72ef58c151e None admin] RBAC: Authorizing `identity:get_project(project_id=0bfd7af746994807b3d8de6f7292ef02)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:04.552500 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45d3360-36c9-46e4-be74-d72ef58c151e None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:04.554127 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 111/223] 10.222.0.18 () {56 vars in 1186 bytes} [Sat Aug 6 21:50:04 2022] GET /identity/v3/projects/0bfd7af746994807b3d8de6f7292ef02 => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:04.554867 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45d3360-36c9-46e4-be74-d72ef58c151e None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:04.627068 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cefd4a-1a88-4cb1-b5c9-20b650f7106d None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:04.655566 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45d3360-36c9-46e4-be74-d72ef58c151e None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:04.660023 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bcb83c-07c0-468f-9fed-2f11f0a9a2cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:04.660680 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bcb83c-07c0-468f-9fed-2f11f0a9a2cc None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:04.660963 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bcb83c-07c0-468f-9fed-2f11f0a9a2cc None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:04.661239 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bcb83c-07c0-468f-9fed-2f11f0a9a2cc None admin] PATH_INFO: `/v3/users/57cdf1d257184e108edfe80e84eb29db` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:04.662363 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bcb83c-07c0-468f-9fed-2f11f0a9a2cc None admin] RBAC: Authorizing `identity:get_user(user_id=57cdf1d257184e108edfe80e84eb29db)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:04.663813 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bcb83c-07c0-468f-9fed-2f11f0a9a2cc None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:04.665434 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 113/224] 10.222.0.18 () {56 vars in 1177 bytes} [Sat Aug 6 21:50:04 2022] GET /identity/v3/users/57cdf1d257184e108edfe80e84eb29db => generated 283 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:04.666204 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bcb83c-07c0-468f-9fed-2f11f0a9a2cc None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:04.689635 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b177ce3-6f0b-42f2-84d8-5a03de2999ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:04.690174 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b177ce3-6f0b-42f2-84d8-5a03de2999ec None admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:04.690632 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b177ce3-6f0b-42f2-84d8-5a03de2999ec None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:04.690987 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b177ce3-6f0b-42f2-84d8-5a03de2999ec None admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:04.691639 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b177ce3-6f0b-42f2-84d8-5a03de2999ec None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:04.693108 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b177ce3-6f0b-42f2-84d8-5a03de2999ec None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:04.745199 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:04.762062 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bcb83c-07c0-468f-9fed-2f11f0a9a2cc None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:04.795509 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71973cd8-59fe-4db6-916a-274d285fd7c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:04.796162 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71973cd8-59fe-4db6-916a-274d285fd7c5 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:04.796465 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71973cd8-59fe-4db6-916a-274d285fd7c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:04.796743 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71973cd8-59fe-4db6-916a-274d285fd7c5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:04.803085 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 112/225] 10.222.0.18 () {60 vars in 1132 bytes} [Sat Aug 6 21:50:04 2022] POST /identity/v3/users => generated 297 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 21:50:04.803907 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b177ce3-6f0b-42f2-84d8-5a03de2999ec None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:04.818425 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71973cd8-59fe-4db6-916a-274d285fd7c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:50:04.823416 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-71973cd8-59fe-4db6-916a-274d285fd7c5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 21:50:04.823416 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:04.823416 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:04.823416 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:04.823416 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:04.823416 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:04.823416 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:04.823416 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:04.823416 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:04.823416 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:04.823416 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:04.823416 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:04.823416 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:50:04.823416 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:50:04.823416 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:50:04.823416 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:50:04.824451 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:50:04.824451 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:50:04.824451 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:50:04.824451 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:50:04.824451 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:04.824451 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:04.824451 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:04.824451 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:04.824451 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:04.824451 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:04.824451 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:04.824451 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:04.824451 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:04.824451 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:04.824451 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:04.824451 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:04.825291 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:04.825291 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:04.825291 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:04.825291 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:04.825291 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:50:04.825291 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:50:04.825291 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:50:04.825291 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:50:04.825291 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:50:04.825291 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:50:04.825291 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 21:50:04.825291 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:50:04.825291 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 114/226] 10.222.0.18 () {56 vars in 1099 bytes} [Sat Aug 6 21:50:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:04.825944 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71973cd8-59fe-4db6-916a-274d285fd7c5 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:04.849456 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:04.929539 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b177ce3-6f0b-42f2-84d8-5a03de2999ec None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:04.941634 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71973cd8-59fe-4db6-916a-274d285fd7c5 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:04.972248 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f26185-cf85-4e7f-8aac-dd4ba728f5db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:04.972955 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f26185-cf85-4e7f-8aac-dd4ba728f5db None admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:04.973232 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f26185-cf85-4e7f-8aac-dd4ba728f5db None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:04.974105 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f26185-cf85-4e7f-8aac-dd4ba728f5db None admin] PATH_INFO: `/v3/projects/0bfd7af746994807b3d8de6f7292ef02/users/57cdf1d257184e108edfe80e84eb29db/roles/93c058ac49e242fd8517c80c552dea2d` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:04.976048 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f26185-cf85-4e7f-8aac-dd4ba728f5db None admin] RBAC: Authorizing `identity:create_grant(project_id=0bfd7af746994807b3d8de6f7292ef02, user_id=57cdf1d257184e108edfe80e84eb29db, role_id=93c058ac49e242fd8517c80c552dea2d)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:04.977294 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f26185-cf85-4e7f-8aac-dd4ba728f5db None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:04.983867 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395ac839-07d9-4f4e-85f6-2671b74fb44e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:04.984539 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395ac839-07d9-4f4e-85f6-2671b74fb44e None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:04.984850 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395ac839-07d9-4f4e-85f6-2671b74fb44e None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:04.985142 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395ac839-07d9-4f4e-85f6-2671b74fb44e None admin] PATH_INFO: `/v3/projects/0bfd7af746994807b3d8de6f7292ef02` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:04.986258 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395ac839-07d9-4f4e-85f6-2671b74fb44e None admin] RBAC: Authorizing `identity:get_project(project_id=0bfd7af746994807b3d8de6f7292ef02)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:04.987875 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395ac839-07d9-4f4e-85f6-2671b74fb44e None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:04.989527 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 115/227] 10.222.0.18 () {56 vars in 1186 bytes} [Sat Aug 6 21:50:04 2022] GET /identity/v3/projects/0bfd7af746994807b3d8de6f7292ef02 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:04.990279 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395ac839-07d9-4f4e-85f6-2671b74fb44e None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:05.002651 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:05.062893 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 113/228] 10.222.0.18 () {58 vars in 1439 bytes} [Sat Aug 6 21:50:04 2022] PUT /identity/v3/projects/0bfd7af746994807b3d8de6f7292ef02/users/57cdf1d257184e108edfe80e84eb29db/roles/93c058ac49e242fd8517c80c552dea2d => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 21:50:05.066592 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f26185-cf85-4e7f-8aac-dd4ba728f5db None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:05.123547 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395ac839-07d9-4f4e-85f6-2671b74fb44e None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:05.169511 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a3c4aa-6dfb-465f-ba45-079bf4b12910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:05.170468 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a3c4aa-6dfb-465f-ba45-079bf4b12910 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:05.170902 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a3c4aa-6dfb-465f-ba45-079bf4b12910 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:05.171325 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a3c4aa-6dfb-465f-ba45-079bf4b12910 None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:05.172156 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a3c4aa-6dfb-465f-ba45-079bf4b12910 None admin] RBAC: Adding query filter params (name=member) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:05.172641 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a3c4aa-6dfb-465f-ba45-079bf4b12910 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:05.174528 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a3c4aa-6dfb-465f-ba45-079bf4b12910 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:05.186891 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 116/229] 10.222.0.18 () {56 vars in 1101 bytes} [Sat Aug 6 21:50:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:05.194139 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a3c4aa-6dfb-465f-ba45-079bf4b12910 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:05.199741 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:05.208156 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f26185-cf85-4e7f-8aac-dd4ba728f5db None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:05.249048 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a214a00d-cfac-4da0-afec-f88c937c5bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:05.250124 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a214a00d-cfac-4da0-afec-f88c937c5bc5 None admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:05.250588 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a214a00d-cfac-4da0-afec-f88c937c5bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:05.251089 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a214a00d-cfac-4da0-afec-f88c937c5bc5 None admin] PATH_INFO: `/v3/projects/0bfd7af746994807b3d8de6f7292ef02/users/57cdf1d257184e108edfe80e84eb29db/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:05.254681 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a214a00d-cfac-4da0-afec-f88c937c5bc5 None admin] RBAC: Authorizing `identity:create_grant(project_id=0bfd7af746994807b3d8de6f7292ef02, user_id=57cdf1d257184e108edfe80e84eb29db, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:05.256541 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a214a00d-cfac-4da0-afec-f88c937c5bc5 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:05.283690 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:05.314841 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a3c4aa-6dfb-465f-ba45-079bf4b12910 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:05.359121 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851b3c1c-2f0e-475e-9fe0-828b5223ca42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:05.360096 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851b3c1c-2f0e-475e-9fe0-828b5223ca42 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:05.360578 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851b3c1c-2f0e-475e-9fe0-828b5223ca42 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:05.361030 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851b3c1c-2f0e-475e-9fe0-828b5223ca42 None admin] PATH_INFO: `/v3/users/12bfa79d93244fd18b45bc0a88d40ef2` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:05.365616 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 114/230] 10.222.0.18 () {58 vars in 1439 bytes} [Sat Aug 6 21:50:05 2022] PUT /identity/v3/projects/0bfd7af746994807b3d8de6f7292ef02/users/57cdf1d257184e108edfe80e84eb29db/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 21:50:05.378798 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851b3c1c-2f0e-475e-9fe0-828b5223ca42 None admin] RBAC: Authorizing `identity:get_user(user_id=12bfa79d93244fd18b45bc0a88d40ef2)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:05.380762 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851b3c1c-2f0e-475e-9fe0-828b5223ca42 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:05.384452 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 117/231] 10.222.0.18 () {56 vars in 1177 bytes} [Sat Aug 6 21:50:05 2022] GET /identity/v3/users/12bfa79d93244fd18b45bc0a88d40ef2 => generated 297 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:05.389013 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a214a00d-cfac-4da0-afec-f88c937c5bc5 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:05.516361 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a214a00d-cfac-4da0-afec-f88c937c5bc5 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:05.555752 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe33ced-a982-4f7e-af29-f472764316ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:05.556428 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe33ced-a982-4f7e-af29-f472764316ca None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:05.556709 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe33ced-a982-4f7e-af29-f472764316ca None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:05.556979 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe33ced-a982-4f7e-af29-f472764316ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:05.584290 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:05.636720 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe33ced-a982-4f7e-af29-f472764316ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:05.638270 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe33ced-a982-4f7e-af29-f472764316ca None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:05.747388 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 115/232] 10.222.0.18 () {58 vars in 1282 bytes} [Sat Aug 6 21:50:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 21:50:05.753658 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851b3c1c-2f0e-475e-9fe0-828b5223ca42 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:05.891686 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851b3c1c-2f0e-475e-9fe0-828b5223ca42 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:05.922572 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:05.929546 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b6c8cf-5d98-42ec-b854-a498b09986b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:05.930181 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b6c8cf-5d98-42ec-b854-a498b09986b3 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:05.930463 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b6c8cf-5d98-42ec-b854-a498b09986b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:05.930781 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b6c8cf-5d98-42ec-b854-a498b09986b3 None admin] PATH_INFO: `/v3/projects/0bfd7af746994807b3d8de6f7292ef02` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:05.931919 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b6c8cf-5d98-42ec-b854-a498b09986b3 None admin] RBAC: Authorizing `identity:get_project(project_id=0bfd7af746994807b3d8de6f7292ef02)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:05.933556 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b6c8cf-5d98-42ec-b854-a498b09986b3 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:05.940881 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 118/233] 10.222.0.18 () {56 vars in 1186 bytes} [Sat Aug 6 21:50:05 2022] GET /identity/v3/projects/0bfd7af746994807b3d8de6f7292ef02 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:05.941254 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe33ced-a982-4f7e-af29-f472764316ca None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:06.023010 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:06.088434 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe33ced-a982-4f7e-af29-f472764316ca None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:06.140711 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2fae17-77d4-4525-8418-dd6b053fd9a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:06.141321 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2fae17-77d4-4525-8418-dd6b053fd9a6 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:06.141597 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2fae17-77d4-4525-8418-dd6b053fd9a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:06.141919 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2fae17-77d4-4525-8418-dd6b053fd9a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:06.142599 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2fae17-77d4-4525-8418-dd6b053fd9a6 None admin] RBAC: Adding query filter params (role.id=56e4a96094e9434cbbafb22579c034f5, scope.project.id=0bfd7af746994807b3d8de6f7292ef02, user.id=12bfa79d93244fd18b45bc0a88d40ef2) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:06.142903 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2fae17-77d4-4525-8418-dd6b053fd9a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:06.144365 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2fae17-77d4-4525-8418-dd6b053fd9a6 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:06.151070 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 116/234] 10.222.0.18 () {56 vars in 1374 bytes} [Sat Aug 6 21:50:05 2022] GET /identity/v3/role_assignments?user.id=12bfa79d93244fd18b45bc0a88d40ef2&scope.project.id=0bfd7af746994807b3d8de6f7292ef02&role.id=56e4a96094e9434cbbafb22579c034f5 => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:07.365113 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7c4542-9ef2-4087-87ae-b72ae247a1eb None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:07.365595 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7c4542-9ef2-4087-87ae-b72ae247a1eb None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:07.365722 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7c4542-9ef2-4087-87ae-b72ae247a1eb None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:07.367653 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 119/235] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:50:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:50:07.372964 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b28c5ae-fa6b-41a5-a8aa-93e21bca5132 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:07.373377 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b28c5ae-fa6b-41a5-a8aa-93e21bca5132 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:07.373546 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b28c5ae-fa6b-41a5-a8aa-93e21bca5132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:07.413965 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-3b28c5ae-fa6b-41a5-a8aa-93e21bca5132 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:50:07.415171 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-3b28c5ae-fa6b-41a5-a8aa-93e21bca5132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:50:07.560161 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 117/236] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:50:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 21:50:07.597714 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7c4542-9ef2-4087-87ae-b72ae247a1eb None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:07.697333 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a95c6b6-3154-4a2d-8219-98c707177bfe None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:07.697524 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a95c6b6-3154-4a2d-8219-98c707177bfe None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:07.697804 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a95c6b6-3154-4a2d-8219-98c707177bfe None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:07.699166 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 118/237] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:50:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:50:07.712437 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3c2b3c-a84a-4f44-9e80-d47517c00620 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:07.713038 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3c2b3c-a84a-4f44-9e80-d47517c00620 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:07.713534 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3c2b3c-a84a-4f44-9e80-d47517c00620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:07.755455 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-bb3c2b3c-a84a-4f44-9e80-d47517c00620 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:50:07.757294 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-bb3c2b3c-a84a-4f44-9e80-d47517c00620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:50:07.775056 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7c4542-9ef2-4087-87ae-b72ae247a1eb None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:07.832156 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:07.836991 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea208c60-49f9-44f0-85f5-73c0b9ff2677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:07.837877 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea208c60-49f9-44f0-85f5-73c0b9ff2677 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:07.838302 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea208c60-49f9-44f0-85f5-73c0b9ff2677 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:07.838708 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea208c60-49f9-44f0-85f5-73c0b9ff2677 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:07.850161 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea208c60-49f9-44f0-85f5-73c0b9ff2677 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:50:07.864271 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-ea208c60-49f9-44f0-85f5-73c0b9ff2677 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 21:50:07.864271 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:07.864271 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:07.864271 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:07.864271 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:07.864271 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:07.864271 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:07.864271 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:07.864271 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:07.864271 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:07.864271 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:07.864271 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:07.864271 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:50:07.864271 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:50:07.864271 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:50:07.864271 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:50:07.865199 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:50:07.865199 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:50:07.865199 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:50:07.865199 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:50:07.865199 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:07.865199 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:07.865199 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:07.865199 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:07.865199 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:07.865199 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:07.865199 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:07.865199 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:07.865199 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:07.865199 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:07.865199 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:07.865199 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:07.866017 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:07.866017 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:07.866017 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:07.866017 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:07.866017 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:50:07.866017 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:50:07.866017 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:50:07.866017 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:50:07.866017 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:50:07.866017 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:50:07.866017 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 21:50:07.866017 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:50:07.866017 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 120/238] 10.222.0.18 () {56 vars in 1099 bytes} [Sat Aug 6 21:50:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:07.866017 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3fd1ff-e144-41f4-acc5-1bcb3d6641d7 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:07.866719 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3fd1ff-e144-41f4-acc5-1bcb3d6641d7 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:07.866719 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3fd1ff-e144-41f4-acc5-1bcb3d6641d7 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:07.866719 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 121/239] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:50:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:50:07.866894 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 119/240] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:50:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 21:50:07.872489 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3c2b3c-a84a-4f44-9e80-d47517c00620 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:07.873280 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d107a54-4ad9-4f25-8d9f-07c499e6ab30 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:07.873769 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d107a54-4ad9-4f25-8d9f-07c499e6ab30 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:07.874218 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d107a54-4ad9-4f25-8d9f-07c499e6ab30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:07.915123 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-1d107a54-4ad9-4f25-8d9f-07c499e6ab30 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:50:07.916888 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-1d107a54-4ad9-4f25-8d9f-07c499e6ab30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:50:08.009820 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:08.044425 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 122/241] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:50:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 21:50:08.044425 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d107a54-4ad9-4f25-8d9f-07c499e6ab30 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:08.075881 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3c2b3c-a84a-4f44-9e80-d47517c00620 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:08.116561 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb57f965-2d9c-4892-acb7-184d77c5dcf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:08.117557 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb57f965-2d9c-4892-acb7-184d77c5dcf8 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:08.118029 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb57f965-2d9c-4892-acb7-184d77c5dcf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:08.118488 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb57f965-2d9c-4892-acb7-184d77c5dcf8 None admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:08.119429 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb57f965-2d9c-4892-acb7-184d77c5dcf8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:08.119918 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb57f965-2d9c-4892-acb7-184d77c5dcf8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:08.121548 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb57f965-2d9c-4892-acb7-184d77c5dcf8 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:08.130747 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 120/242] 10.222.0.18 () {56 vars in 1101 bytes} [Sat Aug 6 21:50:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:08.132014 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb57f965-2d9c-4892-acb7-184d77c5dcf8 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:08.171946 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d107a54-4ad9-4f25-8d9f-07c499e6ab30 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:08.179432 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:08.212051 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f757c81-0693-4b13-99ff-caf417a869e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:08.212719 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f757c81-0693-4b13-99ff-caf417a869e1 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:08.213003 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f757c81-0693-4b13-99ff-caf417a869e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:08.213279 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f757c81-0693-4b13-99ff-caf417a869e1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:08.214403 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f757c81-0693-4b13-99ff-caf417a869e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:08.215877 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f757c81-0693-4b13-99ff-caf417a869e1 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:08.217639 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 123/243] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:50:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:08.218248 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f757c81-0693-4b13-99ff-caf417a869e1 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:08.248226 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb57f965-2d9c-4892-acb7-184d77c5dcf8 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:08.282526 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b8f665-c5d6-4f61-b61a-114f12ad9acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:08.283077 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b8f665-c5d6-4f61-b61a-114f12ad9acb None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:08.283352 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b8f665-c5d6-4f61-b61a-114f12ad9acb None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:08.283617 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b8f665-c5d6-4f61-b61a-114f12ad9acb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:08.284311 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:08.293815 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b8f665-c5d6-4f61-b61a-114f12ad9acb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:50:08.298932 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-09b8f665-c5d6-4f61-b61a-114f12ad9acb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 21:50:08.298932 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:08.298932 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:08.298932 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:08.298932 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:08.298932 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:08.298932 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:08.298932 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:08.298932 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:08.298932 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:08.298932 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:08.298932 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:08.298932 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:50:08.298932 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:50:08.298932 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:50:08.298932 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:50:08.300954 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:50:08.300954 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:50:08.300954 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:50:08.300954 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:50:08.300954 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:08.300954 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:08.300954 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:08.300954 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:08.300954 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:08.300954 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:08.300954 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:08.300954 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:08.300954 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:08.300954 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:08.300954 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:08.300954 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:08.301741 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:08.301741 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:08.301741 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:08.301741 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:08.301741 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:50:08.301741 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:50:08.301741 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:50:08.301741 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:50:08.301741 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:50:08.301741 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:50:08.301741 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 21:50:08.301741 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:50:08.301741 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 121/244] 10.222.0.18 () {56 vars in 1114 bytes} [Sat Aug 6 21:50:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:08.301741 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147e780b-3143-4dd0-8ace-b43a966e4d85 None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:08.302450 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147e780b-3143-4dd0-8ace-b43a966e4d85 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:08.302450 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147e780b-3143-4dd0-8ace-b43a966e4d85 None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:08.303002 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 122/245] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:50:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:50:08.303636 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147e780b-3143-4dd0-8ace-b43a966e4d85 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:08.336396 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f757c81-0693-4b13-99ff-caf417a869e1 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:08.375173 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297a009c-9fd8-4d5a-91b0-100e9760f35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:08.375805 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297a009c-9fd8-4d5a-91b0-100e9760f35a None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:08.376092 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297a009c-9fd8-4d5a-91b0-100e9760f35a None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:08.376364 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297a009c-9fd8-4d5a-91b0-100e9760f35a None admin] PATH_INFO: `/v3/users/ad137aa4ac2d4fb5a7fb4f7fa07fa785` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:08.392075 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297a009c-9fd8-4d5a-91b0-100e9760f35a None admin] RBAC: Authorizing `identity:get_user(user_id=ad137aa4ac2d4fb5a7fb4f7fa07fa785)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:08.394242 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297a009c-9fd8-4d5a-91b0-100e9760f35a None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:08.396178 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 124/246] 10.222.0.18 () {56 vars in 1177 bytes} [Sat Aug 6 21:50:08 2022] GET /identity/v3/users/ad137aa4ac2d4fb5a7fb4f7fa07fa785 => generated 297 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:08.397204 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdc9236-80f6-4633-b9b4-d6e67dc404d9 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:08.397484 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdc9236-80f6-4633-b9b4-d6e67dc404d9 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:08.397755 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdc9236-80f6-4633-b9b4-d6e67dc404d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:08.415890 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-afdc9236-80f6-4633-b9b4-d6e67dc404d9 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:50:08.417139 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-afdc9236-80f6-4633-b9b4-d6e67dc404d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:50:08.418066 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:08.423241 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147e780b-3143-4dd0-8ace-b43a966e4d85 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:08.459187 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069dc2e2-e5dd-4780-b5d9-a93838cceaaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:08.459801 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069dc2e2-e5dd-4780-b5d9-a93838cceaaa None admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:08.460083 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069dc2e2-e5dd-4780-b5d9-a93838cceaaa None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:08.460357 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069dc2e2-e5dd-4780-b5d9-a93838cceaaa None admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:08.461039 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069dc2e2-e5dd-4780-b5d9-a93838cceaaa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:08.462525 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069dc2e2-e5dd-4780-b5d9-a93838cceaaa None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:08.490976 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 125/247] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:50:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 21:50:08.491839 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdc9236-80f6-4633-b9b4-d6e67dc404d9 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:08.537931 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 123/248] 10.222.0.18 () {60 vars in 1132 bytes} [Sat Aug 6 21:50:08 2022] POST /identity/v3/users => generated 293 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 21:50:08.538863 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069dc2e2-e5dd-4780-b5d9-a93838cceaaa None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:08.545408 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:08.618610 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdc9236-80f6-4633-b9b4-d6e67dc404d9 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:08.666748 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a27306-762f-4bb0-a583-4a805667e5f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:08.667378 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a27306-762f-4bb0-a583-4a805667e5f9 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:08.667659 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a27306-762f-4bb0-a583-4a805667e5f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:08.667935 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a27306-762f-4bb0-a583-4a805667e5f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:08.669209 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a27306-762f-4bb0-a583-4a805667e5f9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:08.669731 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a27306-762f-4bb0-a583-4a805667e5f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:08.671454 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a27306-762f-4bb0-a583-4a805667e5f9 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:08.681833 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069dc2e2-e5dd-4780-b5d9-a93838cceaaa None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:08.688588 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 126/249] 10.222.0.18 () {56 vars in 1111 bytes} [Sat Aug 6 21:50:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:08.689445 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a27306-762f-4bb0-a583-4a805667e5f9 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:08.726530 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e062ccc5-d60b-47b4-9319-adfc41473858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:08.727198 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e062ccc5-d60b-47b4-9319-adfc41473858 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:08.727479 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e062ccc5-d60b-47b4-9319-adfc41473858 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:08.727748 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e062ccc5-d60b-47b4-9319-adfc41473858 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:08.755159 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:08.810626 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e062ccc5-d60b-47b4-9319-adfc41473858 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:08.810626 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e062ccc5-d60b-47b4-9319-adfc41473858 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:08.826586 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a27306-762f-4bb0-a583-4a805667e5f9 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:08.836698 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:08.871373 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f6e375-afb8-4ff3-95b6-b0abe51b8cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:08.872601 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f6e375-afb8-4ff3-95b6-b0abe51b8cc1 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:08.873042 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f6e375-afb8-4ff3-95b6-b0abe51b8cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:08.873764 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f6e375-afb8-4ff3-95b6-b0abe51b8cc1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:08.887366 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f6e375-afb8-4ff3-95b6-b0abe51b8cc1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:50:08.892797 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-85f6e375-afb8-4ff3-95b6-b0abe51b8cc1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 21:50:08.892797 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:08.892797 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:08.892797 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:08.892797 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:08.892797 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:08.892797 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:08.892797 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:08.892797 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:08.892797 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:08.892797 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:08.892797 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:08.892797 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:50:08.892797 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:50:08.892797 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:50:08.892797 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:50:08.894345 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:50:08.894345 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:50:08.894345 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:50:08.894345 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:50:08.894345 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:08.894345 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:08.894345 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:08.894345 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:08.894345 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:08.894345 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:08.894345 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:08.894345 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:08.894345 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:08.894345 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:08.894345 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:08.894345 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:08.895217 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:08.895217 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:08.895217 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:08.895217 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:08.895217 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:50:08.895217 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:50:08.895217 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:50:08.895217 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:50:08.895217 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:50:08.895217 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:50:08.895217 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 21:50:08.895217 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:50:08.895217 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 127/250] 10.222.0.18 () {56 vars in 1096 bytes} [Sat Aug 6 21:50:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:08.895859 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f6e375-afb8-4ff3-95b6-b0abe51b8cc1 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:08.942462 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 124/251] 10.222.0.18 () {58 vars in 1282 bytes} [Sat Aug 6 21:50:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 21:50:08.943759 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa5a864-3e5a-4144-8d81-e354f2cc6a20 None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:08.944046 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa5a864-3e5a-4144-8d81-e354f2cc6a20 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:08.944317 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa5a864-3e5a-4144-8d81-e354f2cc6a20 None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:08.945876 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 125/252] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:50:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:50:08.946587 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa5a864-3e5a-4144-8d81-e354f2cc6a20 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:09.020688 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:09.026231 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f6e375-afb8-4ff3-95b6-b0abe51b8cc1 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:09.066877 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cc613a-3ae8-4350-8651-dae2fa2a40b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:09.067516 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cc613a-3ae8-4350-8651-dae2fa2a40b4 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:09.067804 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cc613a-3ae8-4350-8651-dae2fa2a40b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:09.068085 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cc613a-3ae8-4350-8651-dae2fa2a40b4 None admin] PATH_INFO: `/v3/users/57cdf1d257184e108edfe80e84eb29db` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:09.069368 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cc613a-3ae8-4350-8651-dae2fa2a40b4 None admin] RBAC: Authorizing `identity:get_user(user_id=57cdf1d257184e108edfe80e84eb29db)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:09.071010 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cc613a-3ae8-4350-8651-dae2fa2a40b4 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:09.072990 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 128/253] 10.222.0.18 () {56 vars in 1177 bytes} [Sat Aug 6 21:50:08 2022] GET /identity/v3/users/57cdf1d257184e108edfe80e84eb29db => generated 283 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:09.078010 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cc613a-3ae8-4350-8651-dae2fa2a40b4 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:09.080387 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa5a864-3e5a-4144-8d81-e354f2cc6a20 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:09.123959 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9766d557-39ae-4b4a-a3a0-98bbcc169181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:09.124581 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9766d557-39ae-4b4a-a3a0-98bbcc169181 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:09.124903 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9766d557-39ae-4b4a-a3a0-98bbcc169181 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:09.125178 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9766d557-39ae-4b4a-a3a0-98bbcc169181 None admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:09.125853 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9766d557-39ae-4b4a-a3a0-98bbcc169181 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:09.126159 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9766d557-39ae-4b4a-a3a0-98bbcc169181 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:09.127578 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9766d557-39ae-4b4a-a3a0-98bbcc169181 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:09.127997 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:09.137289 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 126/254] 10.222.0.18 () {56 vars in 1099 bytes} [Sat Aug 6 21:50:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:09.141746 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21615440-41ef-41e0-9396-090366f4a2b3 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:09.141746 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21615440-41ef-41e0-9396-090366f4a2b3 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:09.141746 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21615440-41ef-41e0-9396-090366f4a2b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:09.190113 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-21615440-41ef-41e0-9396-090366f4a2b3 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:50:09.191702 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-21615440-41ef-41e0-9396-090366f4a2b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:50:09.253484 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cc613a-3ae8-4350-8651-dae2fa2a40b4 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:09.265199 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 127/255] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:50:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 21:50:09.266421 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21615440-41ef-41e0-9396-090366f4a2b3 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:09.291562 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a23d257-2cdc-47fd-ae37-dd3c884b5ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:09.292555 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a23d257-2cdc-47fd-ae37-dd3c884b5ff1 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:09.293027 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a23d257-2cdc-47fd-ae37-dd3c884b5ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:09.293488 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a23d257-2cdc-47fd-ae37-dd3c884b5ff1 None admin] PATH_INFO: `/v3/projects/0bfd7af746994807b3d8de6f7292ef02` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:09.294931 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a23d257-2cdc-47fd-ae37-dd3c884b5ff1 None admin] RBAC: Authorizing `identity:get_project(project_id=0bfd7af746994807b3d8de6f7292ef02)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:09.296633 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a23d257-2cdc-47fd-ae37-dd3c884b5ff1 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:09.298800 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 129/256] 10.222.0.18 () {56 vars in 1186 bytes} [Sat Aug 6 21:50:09 2022] GET /identity/v3/projects/0bfd7af746994807b3d8de6f7292ef02 => generated 302 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:09.300446 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a23d257-2cdc-47fd-ae37-dd3c884b5ff1 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:09.320251 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:09.376636 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21615440-41ef-41e0-9396-090366f4a2b3 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:09.414515 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e80528-4b59-42b3-abba-667c6a258e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:09.415424 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e80528-4b59-42b3-abba-667c6a258e92 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:09.415851 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e80528-4b59-42b3-abba-667c6a258e92 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:09.416329 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e80528-4b59-42b3-abba-667c6a258e92 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:09.427377 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a23d257-2cdc-47fd-ae37-dd3c884b5ff1 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:09.429064 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:09.463432 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d2dd2e-9f01-4804-b01a-5f4c41e6f9a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:09.464575 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d2dd2e-9f01-4804-b01a-5f4c41e6f9a3 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:09.465536 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d2dd2e-9f01-4804-b01a-5f4c41e6f9a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:09.466335 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d2dd2e-9f01-4804-b01a-5f4c41e6f9a3 None admin] PATH_INFO: `/v3/users/57cdf1d257184e108edfe80e84eb29db` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:09.467777 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d2dd2e-9f01-4804-b01a-5f4c41e6f9a3 None admin] RBAC: Authorizing `identity:get_user(user_id=57cdf1d257184e108edfe80e84eb29db)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:09.469452 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d2dd2e-9f01-4804-b01a-5f4c41e6f9a3 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:09.476631 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 130/257] 10.222.0.18 () {56 vars in 1177 bytes} [Sat Aug 6 21:50:09 2022] GET /identity/v3/users/57cdf1d257184e108edfe80e84eb29db => generated 283 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:09.477855 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d2dd2e-9f01-4804-b01a-5f4c41e6f9a3 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:09.494151 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e80528-4b59-42b3-abba-667c6a258e92 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:09.495623 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e80528-4b59-42b3-abba-667c6a258e92 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:09.547577 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:09.587066 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d2dd2e-9f01-4804-b01a-5f4c41e6f9a3 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:09.602143 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 128/258] 10.222.0.18 () {58 vars in 1282 bytes} [Sat Aug 6 21:50:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 21:50:09.602884 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e80528-4b59-42b3-abba-667c6a258e92 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:09.624228 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf02d7a-153d-46a0-8b61-740ee2aa5457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:09.625192 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf02d7a-153d-46a0-8b61-740ee2aa5457 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:09.625706 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf02d7a-153d-46a0-8b61-740ee2aa5457 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:09.626185 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf02d7a-153d-46a0-8b61-740ee2aa5457 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:09.637429 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf02d7a-153d-46a0-8b61-740ee2aa5457 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:50:09.642899 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-9bf02d7a-153d-46a0-8b61-740ee2aa5457 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 21:50:09.642899 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:09.642899 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:09.642899 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:09.642899 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:09.642899 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:09.642899 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:09.642899 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:09.642899 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:09.642899 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:09.642899 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:09.642899 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:09.642899 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:50:09.642899 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:50:09.642899 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:50:09.642899 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:50:09.643837 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:50:09.643837 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:50:09.643837 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:50:09.643837 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:50:09.643837 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:09.643837 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:09.643837 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:09.643837 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:09.643837 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:09.643837 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:09.643837 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:09.643837 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:09.643837 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:09.643837 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:09.643837 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:09.643837 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:09.644688 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:09.644688 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:09.644688 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:09.644688 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:09.644688 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:50:09.644688 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:50:09.644688 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:50:09.644688 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:50:09.644688 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:50:09.644688 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:50:09.644688 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 21:50:09.644688 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:50:09.646910 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 131/259] 10.222.0.18 () {56 vars in 1099 bytes} [Sat Aug 6 21:50:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:09.647879 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf02d7a-153d-46a0-8b61-740ee2aa5457 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:09.680793 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:09.717125 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e80528-4b59-42b3-abba-667c6a258e92 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:09.758476 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28ee77e-a9bb-4e6b-9357-a54f13fb946b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:09.759704 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28ee77e-a9bb-4e6b-9357-a54f13fb946b None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:09.760177 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28ee77e-a9bb-4e6b-9357-a54f13fb946b None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:09.760865 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28ee77e-a9bb-4e6b-9357-a54f13fb946b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:09.762307 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28ee77e-a9bb-4e6b-9357-a54f13fb946b None admin] RBAC: Adding query filter params (role.id=817eabff1ca5470993116be35777000a, scope.project.id=0bfd7af746994807b3d8de6f7292ef02, user.id=ad137aa4ac2d4fb5a7fb4f7fa07fa785) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:09.762887 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28ee77e-a9bb-4e6b-9357-a54f13fb946b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:09.765280 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28ee77e-a9bb-4e6b-9357-a54f13fb946b None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:09.770874 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf02d7a-153d-46a0-8b61-740ee2aa5457 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:09.781391 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 129/260] 10.222.0.18 () {56 vars in 1374 bytes} [Sat Aug 6 21:50:09 2022] GET /identity/v3/role_assignments?user.id=ad137aa4ac2d4fb5a7fb4f7fa07fa785&scope.project.id=0bfd7af746994807b3d8de6f7292ef02&role.id=817eabff1ca5470993116be35777000a => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:09.781391 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28ee77e-a9bb-4e6b-9357-a54f13fb946b None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:09.808662 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:09.850687 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfc8e5a-f1fc-4ee7-a583-af0629a8236d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:09.851242 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfc8e5a-f1fc-4ee7-a583-af0629a8236d None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:09.851525 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfc8e5a-f1fc-4ee7-a583-af0629a8236d None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:09.851800 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfc8e5a-f1fc-4ee7-a583-af0629a8236d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:09.989051 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28ee77e-a9bb-4e6b-9357-a54f13fb946b None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:10.001444 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfc8e5a-f1fc-4ee7-a583-af0629a8236d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:10.003797 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfc8e5a-f1fc-4ee7-a583-af0629a8236d None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:10.035564 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7376f99-47a2-4e9f-9681-366900e384ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:10.036568 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7376f99-47a2-4e9f-9681-366900e384ea None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:10.037565 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7376f99-47a2-4e9f-9681-366900e384ea None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:10.038033 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7376f99-47a2-4e9f-9681-366900e384ea None admin] PATH_INFO: `/v3/projects/0bfd7af746994807b3d8de6f7292ef02` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:10.039354 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7376f99-47a2-4e9f-9681-366900e384ea None admin] RBAC: Authorizing `identity:get_project(project_id=0bfd7af746994807b3d8de6f7292ef02)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:10.041697 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7376f99-47a2-4e9f-9681-366900e384ea None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:10.043523 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 130/261] 10.222.0.18 () {56 vars in 1186 bytes} [Sat Aug 6 21:50:09 2022] GET /identity/v3/projects/0bfd7af746994807b3d8de6f7292ef02 => generated 302 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:10.044544 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7376f99-47a2-4e9f-9681-366900e384ea None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:10.053136 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:10.145386 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:10.162721 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7376f99-47a2-4e9f-9681-366900e384ea None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:10.201679 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b026e3-aab8-477c-8257-bfce835efed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:10.202650 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b026e3-aab8-477c-8257-bfce835efed2 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:10.203119 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b026e3-aab8-477c-8257-bfce835efed2 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:10.203576 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b026e3-aab8-477c-8257-bfce835efed2 None admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:10.204460 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b026e3-aab8-477c-8257-bfce835efed2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:10.205096 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b026e3-aab8-477c-8257-bfce835efed2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:10.207363 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b026e3-aab8-477c-8257-bfce835efed2 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:10.210912 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 132/262] 10.222.0.18 () {58 vars in 1282 bytes} [Sat Aug 6 21:50:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 563 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 21:50:10.211771 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfc8e5a-f1fc-4ee7-a583-af0629a8236d None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:10.217437 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 131/263] 10.222.0.18 () {56 vars in 1101 bytes} [Sat Aug 6 21:50:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:10.218542 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b026e3-aab8-477c-8257-bfce835efed2 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:10.337970 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b026e3-aab8-477c-8257-bfce835efed2 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:10.423278 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09979a60-2130-4abf-867e-519d96595857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:10.424349 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09979a60-2130-4abf-867e-519d96595857 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:10.424953 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09979a60-2130-4abf-867e-519d96595857 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:10.425419 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09979a60-2130-4abf-867e-519d96595857 None admin] PATH_INFO: `/v3/projects/0bfd7af746994807b3d8de6f7292ef02` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:10.428391 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09979a60-2130-4abf-867e-519d96595857 None admin] RBAC: Authorizing `identity:get_project(project_id=0bfd7af746994807b3d8de6f7292ef02)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:10.431589 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09979a60-2130-4abf-867e-519d96595857 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:10.437586 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 132/264] 10.222.0.18 () {56 vars in 1186 bytes} [Sat Aug 6 21:50:10 2022] GET /identity/v3/projects/0bfd7af746994807b3d8de6f7292ef02 => generated 302 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:10.438624 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09979a60-2130-4abf-867e-519d96595857 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:10.526377 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfc8e5a-f1fc-4ee7-a583-af0629a8236d None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:10.528776 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:10.561597 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09979a60-2130-4abf-867e-519d96595857 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:10.608184 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b520c2-333a-4a6f-9161-71cdbe1ed231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:10.608778 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:10.609041 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b520c2-333a-4a6f-9161-71cdbe1ed231 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:10.609323 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b520c2-333a-4a6f-9161-71cdbe1ed231 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:10.610915 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b520c2-333a-4a6f-9161-71cdbe1ed231 None admin] PATH_INFO: `/v3/users/12bfa79d93244fd18b45bc0a88d40ef2` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:10.612595 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b520c2-333a-4a6f-9161-71cdbe1ed231 None admin] RBAC: Authorizing `identity:get_user(user_id=12bfa79d93244fd18b45bc0a88d40ef2)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:10.614054 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b520c2-333a-4a6f-9161-71cdbe1ed231 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:10.618258 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 133/265] 10.222.0.18 () {56 vars in 1177 bytes} [Sat Aug 6 21:50:10 2022] GET /identity/v3/users/12bfa79d93244fd18b45bc0a88d40ef2 => generated 297 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:10.618258 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b520c2-333a-4a6f-9161-71cdbe1ed231 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:10.619176 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d680f6-1b07-425a-af46-cb9bb5fed438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:10.619857 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d680f6-1b07-425a-af46-cb9bb5fed438 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:10.620141 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d680f6-1b07-425a-af46-cb9bb5fed438 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:10.620430 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d680f6-1b07-425a-af46-cb9bb5fed438 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:10.621237 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d680f6-1b07-425a-af46-cb9bb5fed438 None admin] RBAC: Adding query filter params (role.id=93c058ac49e242fd8517c80c552dea2d, scope.project.id=0bfd7af746994807b3d8de6f7292ef02, user.id=57cdf1d257184e108edfe80e84eb29db) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:10.621545 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d680f6-1b07-425a-af46-cb9bb5fed438 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:10.623052 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d680f6-1b07-425a-af46-cb9bb5fed438 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:10.640033 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 133/266] 10.222.0.18 () {56 vars in 1374 bytes} [Sat Aug 6 21:50:10 2022] GET /identity/v3/role_assignments?user.id=57cdf1d257184e108edfe80e84eb29db&scope.project.id=0bfd7af746994807b3d8de6f7292ef02&role.id=93c058ac49e242fd8517c80c552dea2d => generated 608 bytes in 428 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:10.640990 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d680f6-1b07-425a-af46-cb9bb5fed438 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:10.750237 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:10.768909 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b520c2-333a-4a6f-9161-71cdbe1ed231 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:10.809831 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1080a70d-15ed-47ef-a254-f34517ecf9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:10.810744 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1080a70d-15ed-47ef-a254-f34517ecf9cf None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:10.811030 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1080a70d-15ed-47ef-a254-f34517ecf9cf None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:10.811302 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1080a70d-15ed-47ef-a254-f34517ecf9cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:10.812115 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1080a70d-15ed-47ef-a254-f34517ecf9cf None admin] RBAC: Adding query filter params (role.id=4b8dcf0491fd4072acc69fd894bf77d0, scope.project.id=0bfd7af746994807b3d8de6f7292ef02, user.id=57cdf1d257184e108edfe80e84eb29db) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:10.812510 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1080a70d-15ed-47ef-a254-f34517ecf9cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:10.814146 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1080a70d-15ed-47ef-a254-f34517ecf9cf None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:10.821733 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 134/267] 10.222.0.18 () {56 vars in 1374 bytes} [Sat Aug 6 21:50:10 2022] GET /identity/v3/role_assignments?user.id=57cdf1d257184e108edfe80e84eb29db&scope.project.id=0bfd7af746994807b3d8de6f7292ef02&role.id=4b8dcf0491fd4072acc69fd894bf77d0 => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:10.867684 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a1cfd8-7299-441b-a23f-7753ba42b624 None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:10.868133 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a1cfd8-7299-441b-a23f-7753ba42b624 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:10.868559 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a1cfd8-7299-441b-a23f-7753ba42b624 None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:10.871213 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 135/268] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:50:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 21:50:10.878791 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80c924a-9de8-4574-a232-72aac0acb2f2 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:10.879178 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80c924a-9de8-4574-a232-72aac0acb2f2 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:10.879852 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80c924a-9de8-4574-a232-72aac0acb2f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:10.884981 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d680f6-1b07-425a-af46-cb9bb5fed438 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:10.905232 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-e80c924a-9de8-4574-a232-72aac0acb2f2 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:50:10.907601 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-e80c924a-9de8-4574-a232-72aac0acb2f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:50:10.959268 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9303fcb9-52dd-4949-a942-f2101a69b2fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:10.960321 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9303fcb9-52dd-4949-a942-f2101a69b2fe None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:10.960771 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9303fcb9-52dd-4949-a942-f2101a69b2fe None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:10.964608 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9303fcb9-52dd-4949-a942-f2101a69b2fe None admin] PATH_INFO: `/v3/projects/0bfd7af746994807b3d8de6f7292ef02` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:10.967093 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9303fcb9-52dd-4949-a942-f2101a69b2fe None admin] RBAC: Authorizing `identity:get_project(project_id=0bfd7af746994807b3d8de6f7292ef02)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:10.968861 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9303fcb9-52dd-4949-a942-f2101a69b2fe None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:10.975244 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 134/269] 10.222.0.18 () {56 vars in 1186 bytes} [Sat Aug 6 21:50:10 2022] GET /identity/v3/projects/0bfd7af746994807b3d8de6f7292ef02 => generated 302 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:10.978907 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9303fcb9-52dd-4949-a942-f2101a69b2fe None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:11.016071 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 136/270] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:50:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 21:50:11.044251 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80c924a-9de8-4574-a232-72aac0acb2f2 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:11.056690 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:11.126356 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9303fcb9-52dd-4949-a942-f2101a69b2fe None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:11.148863 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:11.168900 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebb0dba-e295-46e8-90a0-18e750af473d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:11.169569 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebb0dba-e295-46e8-90a0-18e750af473d None admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:11.169874 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebb0dba-e295-46e8-90a0-18e750af473d None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:11.170122 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebb0dba-e295-46e8-90a0-18e750af473d None admin] PATH_INFO: `/v3/projects/0bfd7af746994807b3d8de6f7292ef02/users/12bfa79d93244fd18b45bc0a88d40ef2/roles/56e4a96094e9434cbbafb22579c034f5` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:11.172148 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebb0dba-e295-46e8-90a0-18e750af473d None admin] RBAC: Authorizing `identity:create_grant(project_id=0bfd7af746994807b3d8de6f7292ef02, user_id=12bfa79d93244fd18b45bc0a88d40ef2, role_id=56e4a96094e9434cbbafb22579c034f5)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:11.173709 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebb0dba-e295-46e8-90a0-18e750af473d None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:11.187121 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80c924a-9de8-4574-a232-72aac0acb2f2 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:11.223917 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd662293-9544-478f-b1e0-2684ef293a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:11.226588 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd662293-9544-478f-b1e0-2684ef293a13 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:11.226588 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd662293-9544-478f-b1e0-2684ef293a13 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:11.226588 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd662293-9544-478f-b1e0-2684ef293a13 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:11.235921 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd662293-9544-478f-b1e0-2684ef293a13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:50:11.248861 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-dd662293-9544-478f-b1e0-2684ef293a13 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 21:50:11.248861 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:11.248861 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:11.248861 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:11.248861 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:11.248861 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:11.248861 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:11.248861 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:11.248861 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:11.248861 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:11.248861 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:11.248861 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:11.248861 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:50:11.248861 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:50:11.248861 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:50:11.248861 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:50:11.249804 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:50:11.249804 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:50:11.249804 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:50:11.249804 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:50:11.249804 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:11.249804 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:11.249804 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:11.249804 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:11.249804 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:11.249804 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:11.249804 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:11.249804 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:11.249804 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:11.249804 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:11.249804 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:11.249804 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:11.250612 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:11.250612 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:11.250612 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:11.250612 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:11.250612 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:50:11.250612 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:50:11.250612 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:50:11.250612 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:50:11.250612 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:50:11.250612 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:50:11.250612 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 21:50:11.250612 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:50:11.250612 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 137/272] 10.222.0.18 () {56 vars in 1099 bytes} [Sat Aug 6 21:50:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:11.251771 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455cedb1-273a-4f39-87ed-ad6dbb2dfd1e None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:11.251867 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455cedb1-273a-4f39-87ed-ad6dbb2dfd1e None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:11.252435 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455cedb1-273a-4f39-87ed-ad6dbb2dfd1e None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:11.253033 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 135/272] 10.222.0.18 () {58 vars in 1439 bytes} [Sat Aug 6 21:50:10 2022] PUT /identity/v3/projects/0bfd7af746994807b3d8de6f7292ef02/users/12bfa79d93244fd18b45bc0a88d40ef2/roles/56e4a96094e9434cbbafb22579c034f5 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 21:50:11.253702 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 138/273] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:50:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:50:11.259272 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9771e7f7-1a27-4824-9d27-a88d4ddec56d None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:11.259560 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9771e7f7-1a27-4824-9d27-a88d4ddec56d None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:11.261047 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9771e7f7-1a27-4824-9d27-a88d4ddec56d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:11.263460 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455cedb1-273a-4f39-87ed-ad6dbb2dfd1e None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:11.281413 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-9771e7f7-1a27-4824-9d27-a88d4ddec56d None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:50:11.282688 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-9771e7f7-1a27-4824-9d27-a88d4ddec56d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:50:11.382266 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 136/274] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:50:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 21:50:11.408923 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9771e7f7-1a27-4824-9d27-a88d4ddec56d None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:11.419824 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455cedb1-273a-4f39-87ed-ad6dbb2dfd1e None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:11.458613 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:11.464789 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2702f2-51d2-488b-9f2b-1f84419fd4b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:11.470137 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2702f2-51d2-488b-9f2b-1f84419fd4b5 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:11.470137 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2702f2-51d2-488b-9f2b-1f84419fd4b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:11.470137 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2702f2-51d2-488b-9f2b-1f84419fd4b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:11.470137 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2702f2-51d2-488b-9f2b-1f84419fd4b5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:11.470137 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2702f2-51d2-488b-9f2b-1f84419fd4b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:11.470137 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2702f2-51d2-488b-9f2b-1f84419fd4b5 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:11.470543 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:11.483516 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 139/275] 10.222.0.18 () {56 vars in 1101 bytes} [Sat Aug 6 21:50:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:11.485113 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2702f2-51d2-488b-9f2b-1f84419fd4b5 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:11.525672 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9771e7f7-1a27-4824-9d27-a88d4ddec56d None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:11.564463 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a86fa17-ccc5-4709-8c38-1f7524671de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:11.566057 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a86fa17-ccc5-4709-8c38-1f7524671de3 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:11.566506 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a86fa17-ccc5-4709-8c38-1f7524671de3 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:11.567121 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a86fa17-ccc5-4709-8c38-1f7524671de3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:11.568516 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a86fa17-ccc5-4709-8c38-1f7524671de3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:11.570943 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a86fa17-ccc5-4709-8c38-1f7524671de3 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:11.573108 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 137/276] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:50:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:11.584446 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a86fa17-ccc5-4709-8c38-1f7524671de3 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:11.613606 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2702f2-51d2-488b-9f2b-1f84419fd4b5 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:11.668093 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9010d5-d616-4fe8-8c25-e27a33817327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:11.668804 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9010d5-d616-4fe8-8c25-e27a33817327 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:11.669513 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9010d5-d616-4fe8-8c25-e27a33817327 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:11.669791 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9010d5-d616-4fe8-8c25-e27a33817327 None admin] PATH_INFO: `/v3/users/bb55b8478f4e4a6db4efeee6a5265758` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:11.688345 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a86fa17-ccc5-4709-8c38-1f7524671de3 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:11.688539 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9010d5-d616-4fe8-8c25-e27a33817327 None admin] RBAC: Authorizing `identity:get_user(user_id=bb55b8478f4e4a6db4efeee6a5265758)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:11.690455 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9010d5-d616-4fe8-8c25-e27a33817327 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:11.693376 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 140/277] 10.222.0.18 () {56 vars in 1177 bytes} [Sat Aug 6 21:50:11 2022] GET /identity/v3/users/bb55b8478f4e4a6db4efeee6a5265758 => generated 293 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:50:11.698811 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9010d5-d616-4fe8-8c25-e27a33817327 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:11.714127 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:11.730108 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be5e658-6e4f-463a-9c71-d907bd73ad39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:11.730740 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be5e658-6e4f-463a-9c71-d907bd73ad39 None admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:11.733379 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be5e658-6e4f-463a-9c71-d907bd73ad39 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:11.733379 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be5e658-6e4f-463a-9c71-d907bd73ad39 None admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:11.733379 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be5e658-6e4f-463a-9c71-d907bd73ad39 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:11.734606 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be5e658-6e4f-463a-9c71-d907bd73ad39 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:11.759547 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:11.809013 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9010d5-d616-4fe8-8c25-e27a33817327 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:11.827354 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 138/278] 10.222.0.18 () {60 vars in 1132 bytes} [Sat Aug 6 21:50:11 2022] POST /identity/v3/users => generated 293 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 21:50:11.854651 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d02b10-798b-4222-a3f9-31c9312cfa94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:11.855270 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d02b10-798b-4222-a3f9-31c9312cfa94 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:11.855545 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d02b10-798b-4222-a3f9-31c9312cfa94 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:11.855814 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d02b10-798b-4222-a3f9-31c9312cfa94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:11.856600 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d02b10-798b-4222-a3f9-31c9312cfa94 None admin] RBAC: Adding query filter params (role.id=56e4a96094e9434cbbafb22579c034f5, user.id=bb55b8478f4e4a6db4efeee6a5265758, scope.system=all) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:11.856963 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d02b10-798b-4222-a3f9-31c9312cfa94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:11.858850 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d02b10-798b-4222-a3f9-31c9312cfa94 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:11.866201 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 141/279] 10.222.0.18 () {56 vars in 1308 bytes} [Sat Aug 6 21:50:11 2022] GET /identity/v3/role_assignments?user.id=bb55b8478f4e4a6db4efeee6a5265758&scope.system=all&role.id=56e4a96094e9434cbbafb22579c034f5 => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:12.123795 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d022d275-cd29-46c0-a989-8973a288ae56 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:12.124125 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d022d275-cd29-46c0-a989-8973a288ae56 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:12.124293 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d022d275-cd29-46c0-a989-8973a288ae56 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:12.125916 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 139/280] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:50:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:50:12.132775 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f00b8-16bc-47fa-b2d0-4cdd1388b140 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:12.133053 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f00b8-16bc-47fa-b2d0-4cdd1388b140 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:12.133388 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f00b8-16bc-47fa-b2d0-4cdd1388b140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:12.154254 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-f04f00b8-16bc-47fa-b2d0-4cdd1388b140 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:50:12.155435 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-f04f00b8-16bc-47fa-b2d0-4cdd1388b140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:50:12.249451 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 142/281] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:50:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 21:50:12.273683 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d022d275-cd29-46c0-a989-8973a288ae56 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:12.400062 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d022d275-cd29-46c0-a989-8973a288ae56 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:12.434843 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6e94fb-4187-4378-a095-0080af317251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:12.435477 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6e94fb-4187-4378-a095-0080af317251 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:12.435753 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6e94fb-4187-4378-a095-0080af317251 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:12.436022 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6e94fb-4187-4378-a095-0080af317251 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:12.446895 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6e94fb-4187-4378-a095-0080af317251 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:50:12.451544 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-2c6e94fb-4187-4378-a095-0080af317251 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 21:50:12.451544 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:12.451544 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:12.451544 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:12.451544 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:12.451544 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:12.451544 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:12.451544 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:12.451544 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:12.451544 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:12.451544 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:12.451544 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:12.451544 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:50:12.451544 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:50:12.451544 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:50:12.451544 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:50:12.452453 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:50:12.452453 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:50:12.452453 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:50:12.452453 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:50:12.452453 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:12.452453 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:12.452453 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:12.452453 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:12.452453 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:12.452453 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:12.452453 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:12.452453 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:12.452453 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:12.452453 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:12.452453 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:12.452453 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:12.453248 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:12.453248 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:12.453248 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:12.453248 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:12.453248 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:50:12.453248 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:50:12.453248 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:50:12.453248 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:50:12.453248 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:50:12.453248 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:50:12.453248 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 21:50:12.453248 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:50:12.453248 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 140/282] 10.222.0.18 () {56 vars in 1099 bytes} [Sat Aug 6 21:50:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:12.460965 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04f00b8-16bc-47fa-b2d0-4cdd1388b140 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:12.547488 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:12.624433 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04f00b8-16bc-47fa-b2d0-4cdd1388b140 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:12.663946 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bd0c93-a139-4461-a1b6-e51742358dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:12.664501 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bd0c93-a139-4461-a1b6-e51742358dfe None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:12.664783 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bd0c93-a139-4461-a1b6-e51742358dfe None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:12.665057 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bd0c93-a139-4461-a1b6-e51742358dfe None admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:12.665735 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bd0c93-a139-4461-a1b6-e51742358dfe None admin] RBAC: Adding query filter params (name=reader) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:12.666044 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bd0c93-a139-4461-a1b6-e51742358dfe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:12.667428 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bd0c93-a139-4461-a1b6-e51742358dfe None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:12.682817 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 143/283] 10.222.0.18 () {56 vars in 1101 bytes} [Sat Aug 6 21:50:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:12.689343 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6e94fb-4187-4378-a095-0080af317251 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:12.746655 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:12.817170 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6e94fb-4187-4378-a095-0080af317251 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:12.853400 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee527527-85b9-420e-8b7e-724f93a8d2d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:12.854029 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee527527-85b9-420e-8b7e-724f93a8d2d7 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:12.854304 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee527527-85b9-420e-8b7e-724f93a8d2d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:12.854576 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee527527-85b9-420e-8b7e-724f93a8d2d7 None admin] PATH_INFO: `/v3/users/ad137aa4ac2d4fb5a7fb4f7fa07fa785` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:12.855938 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee527527-85b9-420e-8b7e-724f93a8d2d7 None admin] RBAC: Authorizing `identity:get_user(user_id=ad137aa4ac2d4fb5a7fb4f7fa07fa785)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:12.857711 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee527527-85b9-420e-8b7e-724f93a8d2d7 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:12.859634 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 141/284] 10.222.0.18 () {56 vars in 1177 bytes} [Sat Aug 6 21:50:12 2022] GET /identity/v3/users/ad137aa4ac2d4fb5a7fb4f7fa07fa785 => generated 297 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:12.865192 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bd0c93-a139-4461-a1b6-e51742358dfe None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:12.998458 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bd0c93-a139-4461-a1b6-e51742358dfe None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:13.012386 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:13.045299 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65bab94-093e-45bc-a1e1-6da234783851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:13.045931 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65bab94-093e-45bc-a1e1-6da234783851 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:13.046212 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65bab94-093e-45bc-a1e1-6da234783851 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:13.046487 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65bab94-093e-45bc-a1e1-6da234783851 None admin] PATH_INFO: `/v3/projects/0bfd7af746994807b3d8de6f7292ef02` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:13.047677 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65bab94-093e-45bc-a1e1-6da234783851 None admin] RBAC: Authorizing `identity:get_project(project_id=0bfd7af746994807b3d8de6f7292ef02)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:13.049859 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65bab94-093e-45bc-a1e1-6da234783851 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:13.051928 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 144/285] 10.222.0.18 () {56 vars in 1186 bytes} [Sat Aug 6 21:50:12 2022] GET /identity/v3/projects/0bfd7af746994807b3d8de6f7292ef02 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:13.057640 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee527527-85b9-420e-8b7e-724f93a8d2d7 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:13.220773 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee527527-85b9-420e-8b7e-724f93a8d2d7 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:13.226406 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:13.270597 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0995c08e-faed-452f-a747-67c09482112c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:13.271183 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0995c08e-faed-452f-a747-67c09482112c None admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:13.271465 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0995c08e-faed-452f-a747-67c09482112c None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:13.271739 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0995c08e-faed-452f-a747-67c09482112c None admin] PATH_INFO: `/v3/projects/0bfd7af746994807b3d8de6f7292ef02/users/ad137aa4ac2d4fb5a7fb4f7fa07fa785/roles/817eabff1ca5470993116be35777000a` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:13.273723 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0995c08e-faed-452f-a747-67c09482112c None admin] RBAC: Authorizing `identity:create_grant(project_id=0bfd7af746994807b3d8de6f7292ef02, user_id=ad137aa4ac2d4fb5a7fb4f7fa07fa785, role_id=817eabff1ca5470993116be35777000a)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:13.275174 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0995c08e-faed-452f-a747-67c09482112c None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:13.392212 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 142/286] 10.222.0.18 () {58 vars in 1439 bytes} [Sat Aug 6 21:50:13 2022] PUT /identity/v3/projects/0bfd7af746994807b3d8de6f7292ef02/users/ad137aa4ac2d4fb5a7fb4f7fa07fa785/roles/817eabff1ca5470993116be35777000a => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 21:50:13.659684 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d97b2df-2138-4537-9157-f5f0729715d3 None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:13.660165 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d97b2df-2138-4537-9157-f5f0729715d3 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:13.660165 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d97b2df-2138-4537-9157-f5f0729715d3 None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:13.661807 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 145/287] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:50:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:50:13.667620 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2124a0-a9fb-44a1-9d0d-dd513e66ed4a None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:13.668157 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2124a0-a9fb-44a1-9d0d-dd513e66ed4a None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:13.668631 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2124a0-a9fb-44a1-9d0d-dd513e66ed4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:13.690802 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-bd2124a0-a9fb-44a1-9d0d-dd513e66ed4a None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:50:13.692721 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-bd2124a0-a9fb-44a1-9d0d-dd513e66ed4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:50:13.766748 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 143/288] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:50:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 21:50:13.792045 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d97b2df-2138-4537-9157-f5f0729715d3 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:13.945762 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d97b2df-2138-4537-9157-f5f0729715d3 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:13.993284 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6efcc2a-eb64-4b70-9420-fa4381046616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:13.993923 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6efcc2a-eb64-4b70-9420-fa4381046616 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:13.994198 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6efcc2a-eb64-4b70-9420-fa4381046616 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:13.994468 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6efcc2a-eb64-4b70-9420-fa4381046616 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:14.010915 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6efcc2a-eb64-4b70-9420-fa4381046616 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:50:14.015645 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-b6efcc2a-eb64-4b70-9420-fa4381046616 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 21:50:14.015645 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:14.015645 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:14.015645 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:14.015645 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:14.015645 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:14.015645 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:14.015645 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:14.015645 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:14.015645 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:14.015645 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:14.015645 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:14.015645 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:50:14.015645 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:50:14.015645 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:50:14.015645 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:50:14.016566 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:50:14.016566 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:50:14.016566 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:50:14.016566 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:50:14.016566 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:14.016566 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:14.016566 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:14.016566 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:14.016566 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:14.016566 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:14.016566 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:14.016566 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:14.016566 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:14.016566 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:14.016566 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:14.016566 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:14.017361 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:14.017361 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:14.017361 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:14.017361 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:14.017361 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:50:14.017361 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:50:14.017361 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:50:14.017361 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:50:14.017361 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:50:14.017361 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:50:14.017361 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 21:50:14.017361 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:50:14.019877 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 146/289] 10.222.0.18 () {56 vars in 1099 bytes} [Sat Aug 6 21:50:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:14.024911 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2124a0-a9fb-44a1-9d0d-dd513e66ed4a None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:14.170111 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:14.179626 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2124a0-a9fb-44a1-9d0d-dd513e66ed4a None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:14.238278 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff14348-e490-4f29-8b2d-6f1553d3257d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:14.238898 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff14348-e490-4f29-8b2d-6f1553d3257d None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:14.239181 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff14348-e490-4f29-8b2d-6f1553d3257d None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:14.239456 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff14348-e490-4f29-8b2d-6f1553d3257d None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:14.240313 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff14348-e490-4f29-8b2d-6f1553d3257d None admin] RBAC: Adding query filter params (name=member) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:14.240642 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff14348-e490-4f29-8b2d-6f1553d3257d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:14.242103 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff14348-e490-4f29-8b2d-6f1553d3257d None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:14.257000 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 144/290] 10.222.0.18 () {56 vars in 1101 bytes} [Sat Aug 6 21:50:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:14.262517 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6efcc2a-eb64-4b70-9420-fa4381046616 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:14.344710 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:14.388175 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9da9cf-d95e-4ce0-93a2-deb3cae129c1 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:14.388906 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9da9cf-d95e-4ce0-93a2-deb3cae129c1 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:14.389599 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9da9cf-d95e-4ce0-93a2-deb3cae129c1 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:14.392368 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 145/291] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:50:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 21:50:14.397558 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9b77e0-13d7-4bf1-b0a9-acaf2329f166 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:14.398123 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9b77e0-13d7-4bf1-b0a9-acaf2329f166 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:14.398668 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9b77e0-13d7-4bf1-b0a9-acaf2329f166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:14.417858 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-2b9b77e0-13d7-4bf1-b0a9-acaf2329f166 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:50:14.419467 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-2b9b77e0-13d7-4bf1-b0a9-acaf2329f166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:50:14.427660 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6efcc2a-eb64-4b70-9420-fa4381046616 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:14.481207 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc96e986-4861-44d6-8a39-5d59f38048b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:14.481968 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc96e986-4861-44d6-8a39-5d59f38048b1 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:14.482218 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc96e986-4861-44d6-8a39-5d59f38048b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:14.482748 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc96e986-4861-44d6-8a39-5d59f38048b1 None admin] PATH_INFO: `/v3/users/12bfa79d93244fd18b45bc0a88d40ef2` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:14.483945 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc96e986-4861-44d6-8a39-5d59f38048b1 None admin] RBAC: Authorizing `identity:get_user(user_id=12bfa79d93244fd18b45bc0a88d40ef2)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:14.487871 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc96e986-4861-44d6-8a39-5d59f38048b1 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:14.487871 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 147/292] 10.222.0.18 () {56 vars in 1177 bytes} [Sat Aug 6 21:50:14 2022] GET /identity/v3/users/12bfa79d93244fd18b45bc0a88d40ef2 => generated 297 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:50:14.493359 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc96e986-4861-44d6-8a39-5d59f38048b1 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:14.507613 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 146/293] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:50:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 21:50:14.526814 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3baa3a-b6f4-4c00-864b-aa83cf374960 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:14.527121 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3baa3a-b6f4-4c00-864b-aa83cf374960 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:14.527366 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3baa3a-b6f4-4c00-864b-aa83cf374960 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:14.529161 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 147/294] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:50:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:50:14.535285 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44277c74-d396-4371-8cfb-ecc089e2affa None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:14.535569 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44277c74-d396-4371-8cfb-ecc089e2affa None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:14.535844 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44277c74-d396-4371-8cfb-ecc089e2affa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:14.554005 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-44277c74-d396-4371-8cfb-ecc089e2affa None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:50:14.555200 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-44277c74-d396-4371-8cfb-ecc089e2affa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:50:14.616294 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc96e986-4861-44d6-8a39-5d59f38048b1 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:14.619538 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:14.629928 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 148/295] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:50:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 21:50:14.630978 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44277c74-d396-4371-8cfb-ecc089e2affa None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:14.655932 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:14.659121 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71eabf8-cd19-470a-97cd-1416dc2629b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:14.659842 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71eabf8-cd19-470a-97cd-1416dc2629b6 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:14.660359 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71eabf8-cd19-470a-97cd-1416dc2629b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:14.660359 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71eabf8-cd19-470a-97cd-1416dc2629b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:14.729752 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71eabf8-cd19-470a-97cd-1416dc2629b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:14.731674 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71eabf8-cd19-470a-97cd-1416dc2629b6 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:14.743256 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44277c74-d396-4371-8cfb-ecc089e2affa None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:14.781086 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f019f28-9a0e-41f7-90dc-b3acf5c254f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:14.781643 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f019f28-9a0e-41f7-90dc-b3acf5c254f0 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:14.781921 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f019f28-9a0e-41f7-90dc-b3acf5c254f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:14.782189 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f019f28-9a0e-41f7-90dc-b3acf5c254f0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:14.792504 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f019f28-9a0e-41f7-90dc-b3acf5c254f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:50:14.797124 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-4f019f28-9a0e-41f7-90dc-b3acf5c254f0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 21:50:14.797124 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:14.797124 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:14.797124 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:14.797124 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:14.797124 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:14.797124 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:14.797124 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:14.797124 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:14.797124 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:14.797124 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:14.797124 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:14.797124 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:50:14.797124 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:50:14.797124 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:50:14.797124 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:50:14.797999 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:50:14.797999 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:50:14.797999 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:50:14.797999 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:50:14.797999 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:14.797999 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:14.797999 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:14.797999 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:14.797999 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:14.797999 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:14.797999 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:14.797999 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:14.797999 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:14.797999 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:14.797999 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:14.797999 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:14.804463 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:14.804463 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:14.804463 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:14.804463 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:14.804463 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:50:14.804463 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:50:14.804463 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:50:14.804463 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:50:14.804463 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:50:14.804463 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:50:14.804463 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 21:50:14.804463 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:50:14.804463 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 149/296] 10.222.0.18 () {56 vars in 1099 bytes} [Sat Aug 6 21:50:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:14.804463 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abe3a8b-9536-40d2-8bbd-a3b092895f70 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:14.805249 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abe3a8b-9536-40d2-8bbd-a3b092895f70 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:14.805249 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abe3a8b-9536-40d2-8bbd-a3b092895f70 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:14.805249 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 150/297] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:50:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:50:14.805249 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abe3a8b-9536-40d2-8bbd-a3b092895f70 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:14.843982 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 148/298] 10.222.0.18 () {58 vars in 1282 bytes} [Sat Aug 6 21:50:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 06 21:50:14.848755 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71eabf8-cd19-470a-97cd-1416dc2629b6 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:14.851413 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:14.889532 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:14.919821 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abe3a8b-9536-40d2-8bbd-a3b092895f70 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:14.953764 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71eabf8-cd19-470a-97cd-1416dc2629b6 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:14.963703 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a921d0b-b9d6-4db8-a5ae-594c6439045c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:14.964685 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a921d0b-b9d6-4db8-a5ae-594c6439045c None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:14.965119 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a921d0b-b9d6-4db8-a5ae-594c6439045c None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:14.965393 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a921d0b-b9d6-4db8-a5ae-594c6439045c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:14.966497 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a921d0b-b9d6-4db8-a5ae-594c6439045c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:14.967960 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a921d0b-b9d6-4db8-a5ae-594c6439045c None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:14.969545 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 151/299] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:50:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:14.970646 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b1c74f-eff7-4133-9cab-4f532c0d473b None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:14.972089 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b1c74f-eff7-4133-9cab-4f532c0d473b None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:14.972494 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b1c74f-eff7-4133-9cab-4f532c0d473b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:14.990885 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-55b1c74f-eff7-4133-9cab-4f532c0d473b None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:50:14.992070 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-55b1c74f-eff7-4133-9cab-4f532c0d473b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:50:14.998171 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d02f284-828b-4b42-9c59-ca73e2b2f001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:14.998994 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d02f284-828b-4b42-9c59-ca73e2b2f001 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:14.999948 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d02f284-828b-4b42-9c59-ca73e2b2f001 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:15.000387 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d02f284-828b-4b42-9c59-ca73e2b2f001 None admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:15.001269 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d02f284-828b-4b42-9c59-ca73e2b2f001 None admin] RBAC: Adding query filter params (name=member) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:15.002152 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d02f284-828b-4b42-9c59-ca73e2b2f001 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:15.004345 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d02f284-828b-4b42-9c59-ca73e2b2f001 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:15.015710 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 149/300] 10.222.0.18 () {56 vars in 1101 bytes} [Sat Aug 6 21:50:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:15.017600 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d02f284-828b-4b42-9c59-ca73e2b2f001 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:15.062796 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 152/301] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:50:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 21:50:15.063926 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b1c74f-eff7-4133-9cab-4f532c0d473b None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:15.100572 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:15.134782 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d02f284-828b-4b42-9c59-ca73e2b2f001 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:15.151282 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:15.175667 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b1c74f-eff7-4133-9cab-4f532c0d473b None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:15.244760 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52f685d-2e85-4b70-98d2-179445bc02d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:15.244963 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f033f897-3788-4b21-babb-840834199290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:15.245391 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52f685d-2e85-4b70-98d2-179445bc02d4 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:15.245580 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f033f897-3788-4b21-babb-840834199290 None admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:15.245673 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52f685d-2e85-4b70-98d2-179445bc02d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:15.245816 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f033f897-3788-4b21-babb-840834199290 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:15.245961 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52f685d-2e85-4b70-98d2-179445bc02d4 None admin] PATH_INFO: `/v3/projects/0bfd7af746994807b3d8de6f7292ef02` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:15.246103 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f033f897-3788-4b21-babb-840834199290 None admin] PATH_INFO: `/v3/groups` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:15.246688 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f033f897-3788-4b21-babb-840834199290 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:15.247039 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52f685d-2e85-4b70-98d2-179445bc02d4 None admin] RBAC: Authorizing `identity:get_project(project_id=0bfd7af746994807b3d8de6f7292ef02)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:15.248114 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f033f897-3788-4b21-babb-840834199290 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:15.248497 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52f685d-2e85-4b70-98d2-179445bc02d4 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:15.250258 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 150/302] 10.222.0.18 () {56 vars in 1186 bytes} [Sat Aug 6 21:50:15 2022] GET /identity/v3/projects/0bfd7af746994807b3d8de6f7292ef02 => generated 302 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:15.250923 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52f685d-2e85-4b70-98d2-179445bc02d4 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:15.320530 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 153/303] 10.222.0.18 () {60 vars in 1134 bytes} [Sat Aug 6 21:50:15 2022] POST /identity/v3/groups => generated 229 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 21:50:15.320530 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f033f897-3788-4b21-babb-840834199290 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:15.385810 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52f685d-2e85-4b70-98d2-179445bc02d4 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:15.433648 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a984cee-2c4b-449d-b2a8-c69f065b9617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:15.434399 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a984cee-2c4b-449d-b2a8-c69f065b9617 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:15.434679 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a984cee-2c4b-449d-b2a8-c69f065b9617 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:15.434951 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a984cee-2c4b-449d-b2a8-c69f065b9617 None admin] PATH_INFO: `/v3/users/bb55b8478f4e4a6db4efeee6a5265758` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:15.436193 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a984cee-2c4b-449d-b2a8-c69f065b9617 None admin] RBAC: Authorizing `identity:get_user(user_id=bb55b8478f4e4a6db4efeee6a5265758)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:15.437813 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a984cee-2c4b-449d-b2a8-c69f065b9617 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:15.440142 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 151/304] 10.222.0.18 () {56 vars in 1177 bytes} [Sat Aug 6 21:50:15 2022] GET /identity/v3/users/bb55b8478f4e4a6db4efeee6a5265758 => generated 293 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:15.441012 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a984cee-2c4b-449d-b2a8-c69f065b9617 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:15.443789 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:15.453464 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f033f897-3788-4b21-babb-840834199290 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:15.497583 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0caa11-def0-44da-b3d6-b2f8da5062d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:15.498325 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0caa11-def0-44da-b3d6-b2f8da5062d4 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:15.498656 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0caa11-def0-44da-b3d6-b2f8da5062d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:15.498932 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0caa11-def0-44da-b3d6-b2f8da5062d4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:15.501544 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:15.516281 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0caa11-def0-44da-b3d6-b2f8da5062d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:50:15.522197 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-ff0caa11-def0-44da-b3d6-b2f8da5062d4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 21:50:15.522197 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:15.522197 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:15.522197 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:15.522197 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:15.522197 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:15.522197 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:15.522197 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:15.522197 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:15.522197 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:15.522197 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:15.522197 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:15.522197 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:50:15.522197 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:50:15.522197 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:50:15.522197 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:50:15.523318 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:50:15.523318 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:50:15.523318 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:50:15.523318 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:50:15.523318 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:15.523318 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:15.523318 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:15.523318 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:15.523318 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:15.523318 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:15.523318 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:15.523318 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:15.523318 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:15.523318 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:15.523318 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:15.523318 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:15.524838 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:15.524838 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:15.524838 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:15.524838 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:15.524838 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:50:15.524838 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:50:15.524838 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:50:15.524838 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:50:15.524838 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:50:15.524838 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:50:15.524838 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 21:50:15.524838 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:50:15.524838 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 154/305] 10.222.0.18 () {56 vars in 1099 bytes} [Sat Aug 6 21:50:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:15.524838 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0caa11-def0-44da-b3d6-b2f8da5062d4 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:15.571881 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a984cee-2c4b-449d-b2a8-c69f065b9617 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:15.608227 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb71f54-e9c2-4939-82ac-d2255ca27b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:15.608873 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb71f54-e9c2-4939-82ac-d2255ca27b00 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:15.609150 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb71f54-e9c2-4939-82ac-d2255ca27b00 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:15.609422 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb71f54-e9c2-4939-82ac-d2255ca27b00 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:15.610159 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb71f54-e9c2-4939-82ac-d2255ca27b00 None admin] RBAC: Adding query filter params (role.id=56e4a96094e9434cbbafb22579c034f5, scope.project.id=0bfd7af746994807b3d8de6f7292ef02, user.id=12bfa79d93244fd18b45bc0a88d40ef2) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:15.610469 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb71f54-e9c2-4939-82ac-d2255ca27b00 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:15.611995 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb71f54-e9c2-4939-82ac-d2255ca27b00 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:15.619013 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 152/306] 10.222.0.18 () {56 vars in 1374 bytes} [Sat Aug 6 21:50:15 2022] GET /identity/v3/role_assignments?user.id=12bfa79d93244fd18b45bc0a88d40ef2&scope.project.id=0bfd7af746994807b3d8de6f7292ef02&role.id=56e4a96094e9434cbbafb22579c034f5 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:15.619842 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb71f54-e9c2-4939-82ac-d2255ca27b00 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:15.642652 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0caa11-def0-44da-b3d6-b2f8da5062d4 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:15.691050 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5270f4-b3f2-434e-b82f-d4190d31cacf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:15.692347 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5270f4-b3f2-434e-b82f-d4190d31cacf None admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:15.692853 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5270f4-b3f2-434e-b82f-d4190d31cacf None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:15.693323 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5270f4-b3f2-434e-b82f-d4190d31cacf None admin] PATH_INFO: `/v3/system/users/bb55b8478f4e4a6db4efeee6a5265758/roles/56e4a96094e9434cbbafb22579c034f5` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:15.694673 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:15.698048 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5270f4-b3f2-434e-b82f-d4190d31cacf None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=bb55b8478f4e4a6db4efeee6a5265758, role_id=56e4a96094e9434cbbafb22579c034f5)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:15.700838 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5270f4-b3f2-434e-b82f-d4190d31cacf None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:15.749601 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb71f54-e9c2-4939-82ac-d2255ca27b00 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:15.790555 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca9ec3c-5e07-4091-ab34-25aaee03bc8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:15.791586 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca9ec3c-5e07-4091-ab34-25aaee03bc8d None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:15.792102 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca9ec3c-5e07-4091-ab34-25aaee03bc8d None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:15.792665 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca9ec3c-5e07-4091-ab34-25aaee03bc8d None admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:15.793586 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca9ec3c-5e07-4091-ab34-25aaee03bc8d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:15.794113 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca9ec3c-5e07-4091-ab34-25aaee03bc8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:15.795777 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca9ec3c-5e07-4091-ab34-25aaee03bc8d None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:15.807474 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 153/307] 10.222.0.18 () {56 vars in 1101 bytes} [Sat Aug 6 21:50:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:15.813650 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca9ec3c-5e07-4091-ab34-25aaee03bc8d None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:15.816101 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 155/308] 10.222.0.18 () {58 vars in 1334 bytes} [Sat Aug 6 21:50:15 2022] PUT /identity/v3/system/users/bb55b8478f4e4a6db4efeee6a5265758/roles/56e4a96094e9434cbbafb22579c034f5 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 21:50:15.842942 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efe3e48-dbbf-4f88-b2c7-3323c9d43281 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:15.843903 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efe3e48-dbbf-4f88-b2c7-3323c9d43281 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:15.844388 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efe3e48-dbbf-4f88-b2c7-3323c9d43281 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:15.847500 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 156/309] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:50:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:50:15.852816 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b807ec7d-d80d-42e3-872f-dcdb0eb82e8d None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:15.853084 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b807ec7d-d80d-42e3-872f-dcdb0eb82e8d None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:15.853312 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b807ec7d-d80d-42e3-872f-dcdb0eb82e8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:15.882780 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-b807ec7d-d80d-42e3-872f-dcdb0eb82e8d None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:50:15.884057 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-b807ec7d-d80d-42e3-872f-dcdb0eb82e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:50:15.960966 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:15.972333 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca9ec3c-5e07-4091-ab34-25aaee03bc8d None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:15.982030 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 157/310] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:50:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 21:50:16.005201 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b807ec7d-d80d-42e3-872f-dcdb0eb82e8d None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:16.005684 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:16.019653 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc15223-9c55-4bf7-8a74-24926b521e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:16.020314 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc15223-9c55-4bf7-8a74-24926b521e32 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:16.021116 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc15223-9c55-4bf7-8a74-24926b521e32 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:16.021116 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc15223-9c55-4bf7-8a74-24926b521e32 None admin] PATH_INFO: `/v3/users/f22e360a1bb346aba5fb322329f1608c` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:16.037917 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc15223-9c55-4bf7-8a74-24926b521e32 None admin] RBAC: Authorizing `identity:get_user(user_id=f22e360a1bb346aba5fb322329f1608c)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:16.039176 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc15223-9c55-4bf7-8a74-24926b521e32 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:16.041275 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 154/311] 10.222.0.18 () {56 vars in 1177 bytes} [Sat Aug 6 21:50:15 2022] GET /identity/v3/users/f22e360a1bb346aba5fb322329f1608c => generated 293 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:16.047069 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc15223-9c55-4bf7-8a74-24926b521e32 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:16.126768 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b807ec7d-d80d-42e3-872f-dcdb0eb82e8d None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:16.162358 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc15223-9c55-4bf7-8a74-24926b521e32 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:16.163331 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d720bae-e276-4ab3-9f32-5d03a2cf81ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:16.164189 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d720bae-e276-4ab3-9f32-5d03a2cf81ea None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:16.164698 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d720bae-e276-4ab3-9f32-5d03a2cf81ea None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:16.165161 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d720bae-e276-4ab3-9f32-5d03a2cf81ea None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:16.176709 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d720bae-e276-4ab3-9f32-5d03a2cf81ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:50:16.181908 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-1d720bae-e276-4ab3-9f32-5d03a2cf81ea None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 21:50:16.181908 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:16.181908 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:16.181908 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:16.181908 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:16.181908 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:16.181908 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:16.181908 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:16.181908 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:16.181908 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:16.181908 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:16.181908 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:16.181908 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:50:16.181908 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:50:16.181908 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:50:16.181908 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:50:16.182795 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:50:16.182795 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:50:16.182795 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:50:16.182795 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:50:16.182795 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:16.182795 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:16.182795 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:16.182795 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:16.182795 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:16.182795 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:16.182795 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:16.182795 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:16.182795 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:16.182795 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:16.182795 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:16.182795 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:16.183592 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:16.183592 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:16.183592 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:16.183592 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:16.183592 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:50:16.183592 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:50:16.183592 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:50:16.183592 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:50:16.183592 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:50:16.183592 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:50:16.183592 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 21:50:16.183592 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:50:16.186188 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 158/312] 10.222.0.18 () {56 vars in 1099 bytes} [Sat Aug 6 21:50:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:16.192164 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d720bae-e276-4ab3-9f32-5d03a2cf81ea None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:16.200672 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c97bef-2da6-44c0-bcc9-8c4b6c763887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:16.201286 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c97bef-2da6-44c0-bcc9-8c4b6c763887 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:16.201567 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c97bef-2da6-44c0-bcc9-8c4b6c763887 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:16.201839 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c97bef-2da6-44c0-bcc9-8c4b6c763887 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:16.202595 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c97bef-2da6-44c0-bcc9-8c4b6c763887 None admin] RBAC: Adding query filter params (role.id=817eabff1ca5470993116be35777000a, user.id=f22e360a1bb346aba5fb322329f1608c, scope.system=all) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:16.203188 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c97bef-2da6-44c0-bcc9-8c4b6c763887 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:16.204316 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c97bef-2da6-44c0-bcc9-8c4b6c763887 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:16.210930 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 155/313] 10.222.0.18 () {56 vars in 1308 bytes} [Sat Aug 6 21:50:16 2022] GET /identity/v3/role_assignments?user.id=f22e360a1bb346aba5fb322329f1608c&scope.system=all&role.id=817eabff1ca5470993116be35777000a => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:50:16.269860 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:16.309652 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d720bae-e276-4ab3-9f32-5d03a2cf81ea None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:16.348507 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7668001-4bf3-4f85-bc1b-4e73c57e9a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:16.349574 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7668001-4bf3-4f85-bc1b-4e73c57e9a31 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:16.350049 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7668001-4bf3-4f85-bc1b-4e73c57e9a31 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:16.350510 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7668001-4bf3-4f85-bc1b-4e73c57e9a31 None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:16.351386 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7668001-4bf3-4f85-bc1b-4e73c57e9a31 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:16.351910 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7668001-4bf3-4f85-bc1b-4e73c57e9a31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:16.353611 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7668001-4bf3-4f85-bc1b-4e73c57e9a31 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:16.364440 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 159/314] 10.222.0.18 () {56 vars in 1101 bytes} [Sat Aug 6 21:50:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:50:16.369343 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c97bef-2da6-44c0-bcc9-8c4b6c763887 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:16.424855 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:16.478313 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c97bef-2da6-44c0-bcc9-8c4b6c763887 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:16.513659 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c0b7bb-e8df-4320-8b07-a8a284ce2ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:16.514285 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c0b7bb-e8df-4320-8b07-a8a284ce2ac7 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:16.514557 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c0b7bb-e8df-4320-8b07-a8a284ce2ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:16.514823 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c0b7bb-e8df-4320-8b07-a8a284ce2ac7 None admin] PATH_INFO: `/v3/users/ad137aa4ac2d4fb5a7fb4f7fa07fa785` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:16.516013 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c0b7bb-e8df-4320-8b07-a8a284ce2ac7 None admin] RBAC: Authorizing `identity:get_user(user_id=ad137aa4ac2d4fb5a7fb4f7fa07fa785)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:16.517523 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c0b7bb-e8df-4320-8b07-a8a284ce2ac7 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:16.519308 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 156/315] 10.222.0.18 () {56 vars in 1177 bytes} [Sat Aug 6 21:50:16 2022] GET /identity/v3/users/ad137aa4ac2d4fb5a7fb4f7fa07fa785 => generated 297 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:16.525539 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7668001-4bf3-4f85-bc1b-4e73c57e9a31 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:16.635838 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7668001-4bf3-4f85-bc1b-4e73c57e9a31 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:16.652823 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:16.673849 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d928cc-e577-4311-9957-b93548a9c520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:16.674655 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d928cc-e577-4311-9957-b93548a9c520 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:16.674982 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d928cc-e577-4311-9957-b93548a9c520 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:16.675248 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d928cc-e577-4311-9957-b93548a9c520 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:16.745288 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d928cc-e577-4311-9957-b93548a9c520 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:16.746832 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d928cc-e577-4311-9957-b93548a9c520 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:16.846876 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 160/316] 10.222.0.18 () {58 vars in 1282 bytes} [Sat Aug 6 21:50:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 06 21:50:16.852375 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c0b7bb-e8df-4320-8b07-a8a284ce2ac7 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:16.958689 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c0b7bb-e8df-4320-8b07-a8a284ce2ac7 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:16.976952 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:16.995105 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2914eb22-86ea-4b6e-b01b-092f09291157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:16.995737 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2914eb22-86ea-4b6e-b01b-092f09291157 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:16.996011 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2914eb22-86ea-4b6e-b01b-092f09291157 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:16.996277 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2914eb22-86ea-4b6e-b01b-092f09291157 None admin] PATH_INFO: `/v3/projects/0bfd7af746994807b3d8de6f7292ef02` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:16.997432 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2914eb22-86ea-4b6e-b01b-092f09291157 None admin] RBAC: Authorizing `identity:get_project(project_id=0bfd7af746994807b3d8de6f7292ef02)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:16.998930 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2914eb22-86ea-4b6e-b01b-092f09291157 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:17.000867 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 157/317] 10.222.0.18 () {56 vars in 1186 bytes} [Sat Aug 6 21:50:16 2022] GET /identity/v3/projects/0bfd7af746994807b3d8de6f7292ef02 => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:17.006963 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d928cc-e577-4311-9957-b93548a9c520 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:17.028811 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:17.111401 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d928cc-e577-4311-9957-b93548a9c520 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:17.146384 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481e91cb-fbaf-43b8-9028-5853c85f45c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:17.146988 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481e91cb-fbaf-43b8-9028-5853c85f45c1 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:17.147263 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481e91cb-fbaf-43b8-9028-5853c85f45c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:17.147531 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481e91cb-fbaf-43b8-9028-5853c85f45c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:17.148250 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481e91cb-fbaf-43b8-9028-5853c85f45c1 None admin] RBAC: Adding query filter params (role.id=817eabff1ca5470993116be35777000a, scope.project.id=0bfd7af746994807b3d8de6f7292ef02, user.id=ad137aa4ac2d4fb5a7fb4f7fa07fa785) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:17.148582 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481e91cb-fbaf-43b8-9028-5853c85f45c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:17.150078 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481e91cb-fbaf-43b8-9028-5853c85f45c1 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:17.155974 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 161/318] 10.222.0.18 () {56 vars in 1374 bytes} [Sat Aug 6 21:50:17 2022] GET /identity/v3/role_assignments?user.id=ad137aa4ac2d4fb5a7fb4f7fa07fa785&scope.project.id=0bfd7af746994807b3d8de6f7292ef02&role.id=817eabff1ca5470993116be35777000a => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:17.571381 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce29afa5-f40c-4e3b-bd81-0ad060429741 None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:17.571743 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce29afa5-f40c-4e3b-bd81-0ad060429741 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:17.571936 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce29afa5-f40c-4e3b-bd81-0ad060429741 None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:17.573590 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 158/319] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:50:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:50:17.578627 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709b8339-9000-4c80-995a-51df36a6ad9e None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:17.578934 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709b8339-9000-4c80-995a-51df36a6ad9e None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:17.579179 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709b8339-9000-4c80-995a-51df36a6ad9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:17.598299 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-709b8339-9000-4c80-995a-51df36a6ad9e None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:50:17.599479 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-709b8339-9000-4c80-995a-51df36a6ad9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:50:17.668950 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 162/320] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:50:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 21:50:17.691107 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce29afa5-f40c-4e3b-bd81-0ad060429741 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:17.793176 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce29afa5-f40c-4e3b-bd81-0ad060429741 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:17.826688 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bc4816-830e-49e5-805c-6362419c4334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:17.827315 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bc4816-830e-49e5-805c-6362419c4334 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:17.827592 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bc4816-830e-49e5-805c-6362419c4334 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:17.827863 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bc4816-830e-49e5-805c-6362419c4334 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:17.828980 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bc4816-830e-49e5-805c-6362419c4334 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:17.830696 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bc4816-830e-49e5-805c-6362419c4334 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:17.832435 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 159/321] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:50:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:17.837822 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709b8339-9000-4c80-995a-51df36a6ad9e None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:17.840976 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:17.937477 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709b8339-9000-4c80-995a-51df36a6ad9e None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:17.971609 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a99a417-dda8-4b12-8449-c25753fda300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:17.972449 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a99a417-dda8-4b12-8449-c25753fda300 None admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:17.972674 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a99a417-dda8-4b12-8449-c25753fda300 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:17.972939 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a99a417-dda8-4b12-8449-c25753fda300 None admin] PATH_INFO: `/v3/groups` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:17.973569 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a99a417-dda8-4b12-8449-c25753fda300 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:17.975005 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a99a417-dda8-4b12-8449-c25753fda300 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:17.984302 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3419308d-3f59-48a2-8e13-4e5944124aec None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:17.984816 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3419308d-3f59-48a2-8e13-4e5944124aec None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:17.984901 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3419308d-3f59-48a2-8e13-4e5944124aec None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:17.986207 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 160/322] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:50:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 21:50:17.991722 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b66081-3276-4ed4-bfcc-48eb92469227 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:17.992004 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b66081-3276-4ed4-bfcc-48eb92469227 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:17.992276 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b66081-3276-4ed4-bfcc-48eb92469227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:18.010327 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-48b66081-3276-4ed4-bfcc-48eb92469227 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:50:18.011497 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-48b66081-3276-4ed4-bfcc-48eb92469227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:50:18.049350 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 163/323] 10.222.0.18 () {60 vars in 1134 bytes} [Sat Aug 6 21:50:17 2022] POST /identity/v3/groups => generated 226 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 21:50:18.069021 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:18.092474 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 161/324] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:50:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 21:50:18.113888 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a99a417-dda8-4b12-8449-c25753fda300 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:18.183797 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:18.227274 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a99a417-dda8-4b12-8449-c25753fda300 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:18.261981 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf861d20-e554-4518-9d5c-3ece1aa5cd68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:18.262631 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf861d20-e554-4518-9d5c-3ece1aa5cd68 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:18.262889 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf861d20-e554-4518-9d5c-3ece1aa5cd68 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:18.263149 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf861d20-e554-4518-9d5c-3ece1aa5cd68 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:18.274203 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf861d20-e554-4518-9d5c-3ece1aa5cd68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:50:18.278925 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c44c63-fda2-4573-897f-69b8f319a389 None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:18.279213 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c44c63-fda2-4573-897f-69b8f319a389 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:18.279375 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-bf861d20-e554-4518-9d5c-3ece1aa5cd68 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 21:50:18.279375 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:18.279375 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:18.279375 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:18.279375 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:18.279375 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:18.279375 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:18.279375 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:18.279375 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:18.279375 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:18.279375 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:18.279375 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:18.279375 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:50:18.279375 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:50:18.279375 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:50:18.279375 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:50:18.280207 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:50:18.280207 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:50:18.280207 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:50:18.280207 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:50:18.280207 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:18.280207 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:18.280207 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:18.280207 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:18.280207 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:18.280207 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:18.280207 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:18.280207 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:18.280207 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:18.280207 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:18.280207 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:18.280207 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:18.281094 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:18.281094 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:18.281094 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:18.281094 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:18.281094 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:50:18.281094 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:50:18.281094 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:50:18.281094 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:50:18.281094 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:50:18.281094 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:50:18.281094 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 21:50:18.281094 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:50:18.281738 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c44c63-fda2-4573-897f-69b8f319a389 None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:18.281819 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 164/325] 10.222.0.18 () {56 vars in 1099 bytes} [Sat Aug 6 21:50:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:18.281895 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 162/326] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:50:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:50:18.286165 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c44c63-fda2-4573-897f-69b8f319a389 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:18.286385 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee23f5fe-8300-45f7-8ff0-a7ce77683606 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:18.286671 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee23f5fe-8300-45f7-8ff0-a7ce77683606 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:18.286944 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee23f5fe-8300-45f7-8ff0-a7ce77683606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:18.309225 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-ee23f5fe-8300-45f7-8ff0-a7ce77683606 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:50:18.310412 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-ee23f5fe-8300-45f7-8ff0-a7ce77683606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:50:18.398061 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 165/327] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:50:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 21:50:18.422164 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee23f5fe-8300-45f7-8ff0-a7ce77683606 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:18.435733 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c44c63-fda2-4573-897f-69b8f319a389 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:18.485578 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf1b7e7-eb20-4c0f-a8a7-ec26cdb8bec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:18.486122 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf1b7e7-eb20-4c0f-a8a7-ec26cdb8bec4 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:18.486499 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf1b7e7-eb20-4c0f-a8a7-ec26cdb8bec4 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:18.486863 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf1b7e7-eb20-4c0f-a8a7-ec26cdb8bec4 None admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:18.487804 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf1b7e7-eb20-4c0f-a8a7-ec26cdb8bec4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:18.488245 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf1b7e7-eb20-4c0f-a8a7-ec26cdb8bec4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:18.490301 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf1b7e7-eb20-4c0f-a8a7-ec26cdb8bec4 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:18.495613 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:18.503627 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 163/328] 10.222.0.18 () {56 vars in 1101 bytes} [Sat Aug 6 21:50:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:18.509082 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf1b7e7-eb20-4c0f-a8a7-ec26cdb8bec4 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:18.560657 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee23f5fe-8300-45f7-8ff0-a7ce77683606 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:18.575200 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:18.609917 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7510cd-ba77-4036-81a0-ce7556d8ffaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:18.610559 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7510cd-ba77-4036-81a0-ce7556d8ffaf None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:18.610842 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7510cd-ba77-4036-81a0-ce7556d8ffaf None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:18.611688 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7510cd-ba77-4036-81a0-ce7556d8ffaf None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:18.626446 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7510cd-ba77-4036-81a0-ce7556d8ffaf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:50:18.634298 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-8f7510cd-ba77-4036-81a0-ce7556d8ffaf None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 21:50:18.634298 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:18.634298 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:18.634298 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:18.634298 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:18.634298 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:18.634298 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:18.634298 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:18.634298 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:18.634298 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:18.634298 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:18.634298 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:18.634298 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:50:18.634298 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:50:18.634298 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:50:18.634298 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:50:18.635199 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:50:18.635199 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:50:18.635199 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:50:18.635199 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:50:18.635199 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:18.635199 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:18.635199 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:18.635199 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:18.635199 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:18.635199 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:18.635199 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:18.635199 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:18.635199 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:18.635199 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:18.635199 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:18.635199 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:18.635989 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:18.635989 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:18.635989 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:18.635989 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:18.635989 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:50:18.635989 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:50:18.635989 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:50:18.635989 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:50:18.635989 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:50:18.635989 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:50:18.635989 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 21:50:18.635989 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:50:18.636624 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 166/329] 10.222.0.18 () {56 vars in 1099 bytes} [Sat Aug 6 21:50:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 21:50:18.645290 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf1b7e7-eb20-4c0f-a8a7-ec26cdb8bec4 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:18.645447 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7510cd-ba77-4036-81a0-ce7556d8ffaf None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:18.687319 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f48e003-4d5a-4343-bbbf-3436ad5aeedf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:18.687963 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f48e003-4d5a-4343-bbbf-3436ad5aeedf None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:18.688241 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f48e003-4d5a-4343-bbbf-3436ad5aeedf None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:18.688600 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f48e003-4d5a-4343-bbbf-3436ad5aeedf None admin] PATH_INFO: `/v3/users/bb55b8478f4e4a6db4efeee6a5265758` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:18.690208 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f48e003-4d5a-4343-bbbf-3436ad5aeedf None admin] RBAC: Authorizing `identity:get_user(user_id=bb55b8478f4e4a6db4efeee6a5265758)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:18.691652 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f48e003-4d5a-4343-bbbf-3436ad5aeedf None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:18.693811 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 164/330] 10.222.0.18 () {56 vars in 1177 bytes} [Sat Aug 6 21:50:18 2022] GET /identity/v3/users/bb55b8478f4e4a6db4efeee6a5265758 => generated 293 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:18.701247 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f48e003-4d5a-4343-bbbf-3436ad5aeedf None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:18.778616 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7510cd-ba77-4036-81a0-ce7556d8ffaf None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:18.800802 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:18.825276 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f48e003-4d5a-4343-bbbf-3436ad5aeedf None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:18.826358 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fc953d-1efa-487d-a730-548a6d734b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:18.827281 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc953d-1efa-487d-a730-548a6d734b53 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:18.827803 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc953d-1efa-487d-a730-548a6d734b53 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:18.828081 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc953d-1efa-487d-a730-548a6d734b53 None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:18.828854 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fc953d-1efa-487d-a730-548a6d734b53 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:18.829166 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fc953d-1efa-487d-a730-548a6d734b53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:18.832291 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fc953d-1efa-487d-a730-548a6d734b53 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:18.841150 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 167/331] 10.222.0.18 () {56 vars in 1101 bytes} [Sat Aug 6 21:50:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:18.845539 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:18.847308 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fc953d-1efa-487d-a730-548a6d734b53 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:18.873070 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4055c435-2fae-4794-8d78-d5c9331ee9a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:18.873999 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4055c435-2fae-4794-8d78-d5c9331ee9a4 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:18.874449 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4055c435-2fae-4794-8d78-d5c9331ee9a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:18.874896 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4055c435-2fae-4794-8d78-d5c9331ee9a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:18.876606 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4055c435-2fae-4794-8d78-d5c9331ee9a4 None admin] RBAC: Adding query filter params (role.id=56e4a96094e9434cbbafb22579c034f5, user.id=bb55b8478f4e4a6db4efeee6a5265758, scope.system=all) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:18.877127 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4055c435-2fae-4794-8d78-d5c9331ee9a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:18.879458 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4055c435-2fae-4794-8d78-d5c9331ee9a4 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:18.897911 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 165/332] 10.222.0.18 () {56 vars in 1308 bytes} [Sat Aug 6 21:50:18 2022] GET /identity/v3/role_assignments?user.id=bb55b8478f4e4a6db4efeee6a5265758&scope.system=all&role.id=56e4a96094e9434cbbafb22579c034f5 => generated 510 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:18.976504 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fc953d-1efa-487d-a730-548a6d734b53 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:19.018308 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86760a47-7976-4ffb-b501-0611a9d00c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:19.019192 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86760a47-7976-4ffb-b501-0611a9d00c0b None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:19.019476 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86760a47-7976-4ffb-b501-0611a9d00c0b None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:19.019796 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86760a47-7976-4ffb-b501-0611a9d00c0b None admin] PATH_INFO: `/v3/users/f22e360a1bb346aba5fb322329f1608c` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:19.021100 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86760a47-7976-4ffb-b501-0611a9d00c0b None admin] RBAC: Authorizing `identity:get_user(user_id=f22e360a1bb346aba5fb322329f1608c)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:19.022574 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86760a47-7976-4ffb-b501-0611a9d00c0b None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:19.024539 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 168/333] 10.222.0.18 () {56 vars in 1177 bytes} [Sat Aug 6 21:50:18 2022] GET /identity/v3/users/f22e360a1bb346aba5fb322329f1608c => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:50:19.030516 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4055c435-2fae-4794-8d78-d5c9331ee9a4 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:19.147706 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4055c435-2fae-4794-8d78-d5c9331ee9a4 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:19.191020 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbb9229-a200-4c2a-aaec-0f008562b0be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:19.192017 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbb9229-a200-4c2a-aaec-0f008562b0be None admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:19.192512 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbb9229-a200-4c2a-aaec-0f008562b0be None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:19.193221 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbb9229-a200-4c2a-aaec-0f008562b0be None admin] PATH_INFO: `/v3/system/users/f22e360a1bb346aba5fb322329f1608c/roles/817eabff1ca5470993116be35777000a` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:19.195803 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbb9229-a200-4c2a-aaec-0f008562b0be None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=f22e360a1bb346aba5fb322329f1608c, role_id=817eabff1ca5470993116be35777000a)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:19.198254 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbb9229-a200-4c2a-aaec-0f008562b0be None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:19.266498 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 166/334] 10.222.0.18 () {58 vars in 1334 bytes} [Sat Aug 6 21:50:19 2022] PUT /identity/v3/system/users/f22e360a1bb346aba5fb322329f1608c/roles/817eabff1ca5470993116be35777000a => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 21:50:20.751873 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e084de80-01a9-4f5f-92e8-47fc72b14182 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:20.752382 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e084de80-01a9-4f5f-92e8-47fc72b14182 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:20.752680 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e084de80-01a9-4f5f-92e8-47fc72b14182 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:20.753916 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 169/335] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:50:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:50:20.770356 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a20b69-f63a-47f5-8cbb-6b70ac223af5 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:20.770850 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a20b69-f63a-47f5-8cbb-6b70ac223af5 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:20.770924 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a20b69-f63a-47f5-8cbb-6b70ac223af5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:20.797412 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-64a20b69-f63a-47f5-8cbb-6b70ac223af5 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:50:20.798742 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-64a20b69-f63a-47f5-8cbb-6b70ac223af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:50:20.837008 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:20.880291 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 167/336] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:50:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 21:50:20.906758 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4d8209-6625-4e97-ac81-7f9904fd6750 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:20.907060 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4d8209-6625-4e97-ac81-7f9904fd6750 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:20.907316 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4d8209-6625-4e97-ac81-7f9904fd6750 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:20.909261 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 170/337] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:50:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:50:20.919432 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b69858-ef03-4903-9663-127c195f284e None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:20.919601 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4d8209-6625-4e97-ac81-7f9904fd6750 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:20.919794 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b69858-ef03-4903-9663-127c195f284e None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:20.920066 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b69858-ef03-4903-9663-127c195f284e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:20.941903 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-20b69858-ef03-4903-9663-127c195f284e None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:50:20.943126 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-20b69858-ef03-4903-9663-127c195f284e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:50:20.979891 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:21.038792 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4d8209-6625-4e97-ac81-7f9904fd6750 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:21.074944 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ac8b07-d593-4fb9-bbaf-8a05e8fb4700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:21.075577 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ac8b07-d593-4fb9-bbaf-8a05e8fb4700 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:21.075852 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ac8b07-d593-4fb9-bbaf-8a05e8fb4700 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:21.076122 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ac8b07-d593-4fb9-bbaf-8a05e8fb4700 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:21.082801 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 168/338] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:50:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 21:50:21.094136 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ac8b07-d593-4fb9-bbaf-8a05e8fb4700 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:50:21.100251 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-10ac8b07-d593-4fb9-bbaf-8a05e8fb4700 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 21:50:21.100251 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:21.100251 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:21.100251 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:21.100251 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:21.100251 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:21.100251 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:21.100251 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:21.100251 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:21.100251 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:21.100251 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:21.100251 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:21.100251 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:50:21.100251 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:50:21.100251 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:50:21.100251 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:50:21.101152 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:50:21.101152 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:50:21.101152 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:50:21.101152 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:50:21.101152 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:21.101152 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:21.101152 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:21.101152 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:21.101152 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:21.101152 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:21.101152 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:21.101152 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:21.101152 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:21.101152 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:21.101152 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:21.101152 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:21.102375 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:21.102375 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:21.102375 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:21.102375 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:21.102375 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:50:21.102375 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:50:21.102375 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:50:21.102375 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:50:21.102375 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:50:21.102375 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:50:21.102375 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 21:50:21.102375 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:50:21.102375 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 171/339] 10.222.0.18 () {56 vars in 1099 bytes} [Sat Aug 6 21:50:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:21.107827 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b69858-ef03-4903-9663-127c195f284e None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:21.125817 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ac8b07-d593-4fb9-bbaf-8a05e8fb4700 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:21.259918 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b69858-ef03-4903-9663-127c195f284e None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:21.264426 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ac8b07-d593-4fb9-bbaf-8a05e8fb4700 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:21.272765 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:21.273350 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:21.303367 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5395d4ac-4a6e-4abc-981f-c8eb469ae269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:21.304124 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5395d4ac-4a6e-4abc-981f-c8eb469ae269 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:21.305721 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5395d4ac-4a6e-4abc-981f-c8eb469ae269 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:21.306024 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5395d4ac-4a6e-4abc-981f-c8eb469ae269 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:21.325133 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb05fed-f8be-4152-bab5-37077dd55239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:21.325867 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb05fed-f8be-4152-bab5-37077dd55239 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:21.326217 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb05fed-f8be-4152-bab5-37077dd55239 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:21.326560 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb05fed-f8be-4152-bab5-37077dd55239 None admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:21.327350 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb05fed-f8be-4152-bab5-37077dd55239 None admin] RBAC: Adding query filter params (name=member) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:21.327712 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb05fed-f8be-4152-bab5-37077dd55239 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:21.329198 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb05fed-f8be-4152-bab5-37077dd55239 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:21.331322 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5395d4ac-4a6e-4abc-981f-c8eb469ae269 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:50:21.337566 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-5395d4ac-4a6e-4abc-981f-c8eb469ae269 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 21:50:21.337566 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:21.337566 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:21.337566 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:21.337566 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:21.337566 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:21.337566 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:21.337566 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:21.337566 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:21.337566 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:21.337566 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:21.337566 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:21.337566 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:50:21.337566 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:50:21.337566 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:50:21.337566 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:50:21.338495 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:50:21.338495 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:50:21.338495 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:50:21.338495 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:50:21.338495 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:21.338495 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:21.338495 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:21.338495 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:21.338495 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:21.338495 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:21.338495 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:21.338495 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:21.338495 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:21.338495 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:21.338495 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:21.338495 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:21.339267 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:21.339267 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:21.339267 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:21.339267 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:21.339267 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:50:21.339267 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:50:21.339267 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:50:21.339267 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:50:21.339267 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:50:21.339267 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:50:21.339267 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 21:50:21.339267 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:50:21.339267 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 172/340] 10.222.0.18 () {56 vars in 1114 bytes} [Sat Aug 6 21:50:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 21:50:21.345017 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5395d4ac-4a6e-4abc-981f-c8eb469ae269 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:21.354782 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 169/341] 10.222.0.18 () {56 vars in 1101 bytes} [Sat Aug 6 21:50:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:21.359854 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb05fed-f8be-4152-bab5-37077dd55239 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:21.510753 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5395d4ac-4a6e-4abc-981f-c8eb469ae269 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:21.569745 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe485cf-0dbd-41cd-b6a1-048c5e4c0a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:21.570589 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe485cf-0dbd-41cd-b6a1-048c5e4c0a0e None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:21.570876 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe485cf-0dbd-41cd-b6a1-048c5e4c0a0e None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:21.571149 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe485cf-0dbd-41cd-b6a1-048c5e4c0a0e None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:21.571845 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe485cf-0dbd-41cd-b6a1-048c5e4c0a0e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:21.572146 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe485cf-0dbd-41cd-b6a1-048c5e4c0a0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:21.573628 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe485cf-0dbd-41cd-b6a1-048c5e4c0a0e None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:21.592309 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 173/342] 10.222.0.18 () {56 vars in 1111 bytes} [Sat Aug 6 21:50:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:21.599999 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe485cf-0dbd-41cd-b6a1-048c5e4c0a0e None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:21.608483 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb05fed-f8be-4152-bab5-37077dd55239 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:21.646520 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:21.698751 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dbbc58-92f3-4cc0-9ed9-e70f18e73a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:21.699235 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dbbc58-92f3-4cc0-9ed9-e70f18e73a7d None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:21.699510 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dbbc58-92f3-4cc0-9ed9-e70f18e73a7d None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:21.699778 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dbbc58-92f3-4cc0-9ed9-e70f18e73a7d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:21.742013 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe485cf-0dbd-41cd-b6a1-048c5e4c0a0e None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:21.781803 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:21.782305 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8669cffc-a625-45aa-b59f-a9bc9ddc315d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:21.782866 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8669cffc-a625-45aa-b59f-a9bc9ddc315d None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:21.783665 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8669cffc-a625-45aa-b59f-a9bc9ddc315d None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:21.783928 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8669cffc-a625-45aa-b59f-a9bc9ddc315d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:21.849533 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dbbc58-92f3-4cc0-9ed9-e70f18e73a7d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:21.850950 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dbbc58-92f3-4cc0-9ed9-e70f18e73a7d None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:21.923437 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8669cffc-a625-45aa-b59f-a9bc9ddc315d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:21.925737 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8669cffc-a625-45aa-b59f-a9bc9ddc315d None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:21.969245 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:21.981205 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 170/343] 10.222.0.18 () {58 vars in 1282 bytes} [Sat Aug 6 21:50:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 626 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 21:50:21.993461 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dbbc58-92f3-4cc0-9ed9-e70f18e73a7d None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:22.061663 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 174/344] 10.222.0.18 () {58 vars in 1282 bytes} [Sat Aug 6 21:50:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 06 21:50:22.062942 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b7fc8c-9c97-454b-9427-3f864dad1c6f None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:22.063226 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b7fc8c-9c97-454b-9427-3f864dad1c6f None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:22.063499 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b7fc8c-9c97-454b-9427-3f864dad1c6f None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:22.069295 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 175/345] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:50:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:50:22.071638 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b7fc8c-9c97-454b-9427-3f864dad1c6f None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:22.109301 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:22.184243 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b7fc8c-9c97-454b-9427-3f864dad1c6f None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:22.217897 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:22.223191 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02aac40-539f-4c1a-a671-e101919c3d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:22.224251 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02aac40-539f-4c1a-a671-e101919c3d86 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:22.224703 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02aac40-539f-4c1a-a671-e101919c3d86 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:22.224981 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02aac40-539f-4c1a-a671-e101919c3d86 None admin] PATH_INFO: `/v3/projects/edee8005e8714a9db1c10cfec12857b6` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:22.226257 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02aac40-539f-4c1a-a671-e101919c3d86 None admin] RBAC: Authorizing `identity:get_project(project_id=edee8005e8714a9db1c10cfec12857b6)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:22.227989 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02aac40-539f-4c1a-a671-e101919c3d86 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:22.230074 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 176/346] 10.222.0.18 () {56 vars in 1186 bytes} [Sat Aug 6 21:50:22 2022] GET /identity/v3/projects/edee8005e8714a9db1c10cfec12857b6 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:50:22.231737 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b797a6-0d1b-4b20-953e-dc0f1cbb84a7 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:22.232024 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b797a6-0d1b-4b20-953e-dc0f1cbb84a7 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:22.239073 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b797a6-0d1b-4b20-953e-dc0f1cbb84a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:22.242226 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dbbc58-92f3-4cc0-9ed9-e70f18e73a7d None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:22.258967 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-76b797a6-0d1b-4b20-953e-dc0f1cbb84a7 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:50:22.260793 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-76b797a6-0d1b-4b20-953e-dc0f1cbb84a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:50:22.336782 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 177/347] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:50:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 21:50:22.337784 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b797a6-0d1b-4b20-953e-dc0f1cbb84a7 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:22.385298 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1432047a-66c0-460e-8c89-390bb727706d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:22.385871 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1432047a-66c0-460e-8c89-390bb727706d None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:22.386183 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1432047a-66c0-460e-8c89-390bb727706d None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:22.386412 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1432047a-66c0-460e-8c89-390bb727706d None admin] PATH_INFO: `/v3/projects/edee8005e8714a9db1c10cfec12857b6` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:22.387667 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1432047a-66c0-460e-8c89-390bb727706d None admin] RBAC: Authorizing `identity:get_project(project_id=edee8005e8714a9db1c10cfec12857b6)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:22.389140 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1432047a-66c0-460e-8c89-390bb727706d None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:22.391777 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 171/348] 10.222.0.18 () {56 vars in 1186 bytes} [Sat Aug 6 21:50:21 2022] GET /identity/v3/projects/edee8005e8714a9db1c10cfec12857b6 => generated 298 bytes in 398 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:22.396979 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1432047a-66c0-460e-8c89-390bb727706d None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:22.456858 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b797a6-0d1b-4b20-953e-dc0f1cbb84a7 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:22.484894 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:22.497781 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9545578-d0ce-42b2-b8cc-9be9e0c4851e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:22.498482 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9545578-d0ce-42b2-b8cc-9be9e0c4851e None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:22.498760 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9545578-d0ce-42b2-b8cc-9be9e0c4851e None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:22.499030 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9545578-d0ce-42b2-b8cc-9be9e0c4851e None admin] PATH_INFO: `/v3/groups/c415347e96884e65a7ab87b53f4860d2` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:22.506959 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9545578-d0ce-42b2-b8cc-9be9e0c4851e None admin] RBAC: Authorizing `identity:get_group(group_id=c415347e96884e65a7ab87b53f4860d2)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:22.508857 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9545578-d0ce-42b2-b8cc-9be9e0c4851e None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:22.517017 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 178/349] 10.222.0.18 () {56 vars in 1180 bytes} [Sat Aug 6 21:50:22 2022] GET /identity/v3/groups/c415347e96884e65a7ab87b53f4860d2 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:22.518323 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9545578-d0ce-42b2-b8cc-9be9e0c4851e None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:22.617967 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1432047a-66c0-460e-8c89-390bb727706d None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:22.630672 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9545578-d0ce-42b2-b8cc-9be9e0c4851e None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:22.645570 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:22.655040 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48768bd1-12c7-4dd3-b329-8219f4a06c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:22.655675 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48768bd1-12c7-4dd3-b329-8219f4a06c67 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:22.655947 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48768bd1-12c7-4dd3-b329-8219f4a06c67 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:22.656887 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48768bd1-12c7-4dd3-b329-8219f4a06c67 None admin] PATH_INFO: `/v3/groups/c415347e96884e65a7ab87b53f4860d2` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:22.657496 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48768bd1-12c7-4dd3-b329-8219f4a06c67 None admin] RBAC: Authorizing `identity:get_group(group_id=c415347e96884e65a7ab87b53f4860d2)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:22.659598 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48768bd1-12c7-4dd3-b329-8219f4a06c67 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:22.661563 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 172/350] 10.222.0.18 () {56 vars in 1180 bytes} [Sat Aug 6 21:50:22 2022] GET /identity/v3/groups/c415347e96884e65a7ab87b53f4860d2 => generated 226 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:22.662346 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48768bd1-12c7-4dd3-b329-8219f4a06c67 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:22.679369 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f9e320-6041-4051-91ce-d2cd1ebae4f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:22.680011 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f9e320-6041-4051-91ce-d2cd1ebae4f6 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:22.680286 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f9e320-6041-4051-91ce-d2cd1ebae4f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:22.680739 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f9e320-6041-4051-91ce-d2cd1ebae4f6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:22.694496 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f9e320-6041-4051-91ce-d2cd1ebae4f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:50:22.701642 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-72f9e320-6041-4051-91ce-d2cd1ebae4f6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 21:50:22.701642 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:22.701642 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:22.701642 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:22.701642 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:22.701642 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:22.701642 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:22.701642 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:22.701642 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:22.701642 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:22.701642 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:22.701642 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:22.701642 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:50:22.701642 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:50:22.701642 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:50:22.701642 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:50:22.702560 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:50:22.702560 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:50:22.702560 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:50:22.702560 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:50:22.702560 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:22.702560 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:22.702560 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:22.702560 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:22.702560 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:22.702560 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:22.702560 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:22.702560 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:22.702560 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:22.702560 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:22.702560 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:22.702560 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:22.703362 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:22.703362 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:22.703362 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:22.703362 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:22.703362 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:50:22.703362 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:50:22.703362 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:50:22.703362 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:50:22.703362 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:50:22.703362 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:50:22.703362 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 21:50:22.703362 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:50:22.708696 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 179/351] 10.222.0.18 () {56 vars in 1099 bytes} [Sat Aug 6 21:50:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:22.710017 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec48db61-1f63-4211-9927-5ca427bb4e72 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:22.710297 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec48db61-1f63-4211-9927-5ca427bb4e72 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:22.710569 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec48db61-1f63-4211-9927-5ca427bb4e72 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:22.712135 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 180/352] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:50:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:50:22.712924 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec48db61-1f63-4211-9927-5ca427bb4e72 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:22.773834 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:22.783114 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48768bd1-12c7-4dd3-b329-8219f4a06c67 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:22.827087 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8c7df4-fd6e-4734-8f5c-17a37a5de1db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:22.827834 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c7df4-fd6e-4734-8f5c-17a37a5de1db None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:22.828112 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c7df4-fd6e-4734-8f5c-17a37a5de1db None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:22.828386 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c7df4-fd6e-4734-8f5c-17a37a5de1db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:22.829157 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8c7df4-fd6e-4734-8f5c-17a37a5de1db None admin] RBAC: Adding query filter params (group.id=c415347e96884e65a7ab87b53f4860d2, role.id=93c058ac49e242fd8517c80c552dea2d, scope.project.id=edee8005e8714a9db1c10cfec12857b6) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:22.829472 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8c7df4-fd6e-4734-8f5c-17a37a5de1db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:22.831407 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8c7df4-fd6e-4734-8f5c-17a37a5de1db None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:22.839479 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 173/353] 10.222.0.18 () {56 vars in 1376 bytes} [Sat Aug 6 21:50:22 2022] GET /identity/v3/role_assignments?group.id=c415347e96884e65a7ab87b53f4860d2&scope.project.id=edee8005e8714a9db1c10cfec12857b6&role.id=93c058ac49e242fd8517c80c552dea2d => generated 260 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:22.840681 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d951c3-c92b-4764-943b-653630b93192 None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:22.840963 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d951c3-c92b-4764-943b-653630b93192 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:22.841234 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d951c3-c92b-4764-943b-653630b93192 None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:22.841661 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec48db61-1f63-4211-9927-5ca427bb4e72 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:22.842477 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 174/354] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:50:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:50:22.843206 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d951c3-c92b-4764-943b-653630b93192 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:22.882213 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1483e7aa-aca8-4947-9d48-c1bcbfe3f6d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:22.882943 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1483e7aa-aca8-4947-9d48-c1bcbfe3f6d3 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:22.883212 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1483e7aa-aca8-4947-9d48-c1bcbfe3f6d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:22.883574 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1483e7aa-aca8-4947-9d48-c1bcbfe3f6d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:22.884220 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1483e7aa-aca8-4947-9d48-c1bcbfe3f6d3 None admin] RBAC: Adding query filter params (group.id=c415347e96884e65a7ab87b53f4860d2, role.id=56e4a96094e9434cbbafb22579c034f5, scope.project.id=edee8005e8714a9db1c10cfec12857b6) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:22.884616 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1483e7aa-aca8-4947-9d48-c1bcbfe3f6d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:22.886101 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1483e7aa-aca8-4947-9d48-c1bcbfe3f6d3 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:22.894560 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 181/355] 10.222.0.18 () {56 vars in 1376 bytes} [Sat Aug 6 21:50:22 2022] GET /identity/v3/role_assignments?group.id=c415347e96884e65a7ab87b53f4860d2&scope.project.id=edee8005e8714a9db1c10cfec12857b6&role.id=56e4a96094e9434cbbafb22579c034f5 => generated 260 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:22.896510 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8c39bc-6250-4654-a59a-025a0cd994ea None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:22.896510 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8c39bc-6250-4654-a59a-025a0cd994ea None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:22.896510 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8c39bc-6250-4654-a59a-025a0cd994ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:22.919858 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-df8c39bc-6250-4654-a59a-025a0cd994ea None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:50:22.921219 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-df8c39bc-6250-4654-a59a-025a0cd994ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:50:22.933002 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:23.003742 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d951c3-c92b-4764-943b-653630b93192 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:23.021382 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 182/356] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:50:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 21:50:23.022944 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885a4331-c36c-4b38-8744-be4c2350283c None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:23.023298 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885a4331-c36c-4b38-8744-be4c2350283c None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:23.023749 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885a4331-c36c-4b38-8744-be4c2350283c None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:23.025615 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 183/357] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:50:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:50:23.027125 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a704ed-01ce-4454-97ac-eb150b8595ba None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:23.027504 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a704ed-01ce-4454-97ac-eb150b8595ba None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:23.027868 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a704ed-01ce-4454-97ac-eb150b8595ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:23.049656 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-b9a704ed-01ce-4454-97ac-eb150b8595ba None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:50:23.051060 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-b9a704ed-01ce-4454-97ac-eb150b8595ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:50:23.057789 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0fb66f-8e2c-4b87-9001-d6e009083a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:23.058678 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0fb66f-8e2c-4b87-9001-d6e009083a91 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:23.059096 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0fb66f-8e2c-4b87-9001-d6e009083a91 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:23.059505 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0fb66f-8e2c-4b87-9001-d6e009083a91 None admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:23.060326 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0fb66f-8e2c-4b87-9001-d6e009083a91 None admin] RBAC: Adding query filter params (name=member) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:23.060807 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0fb66f-8e2c-4b87-9001-d6e009083a91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:23.063145 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0fb66f-8e2c-4b87-9001-d6e009083a91 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:23.079047 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 175/358] 10.222.0.18 () {56 vars in 1101 bytes} [Sat Aug 6 21:50:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:23.080682 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411c30dc-c8ec-44c9-84ac-273d9b906019 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:23.081106 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411c30dc-c8ec-44c9-84ac-273d9b906019 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:23.081500 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411c30dc-c8ec-44c9-84ac-273d9b906019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:23.104392 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-411c30dc-c8ec-44c9-84ac-273d9b906019 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:50:23.105607 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-411c30dc-c8ec-44c9-84ac-273d9b906019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:50:23.111621 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:23.155725 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 184/359] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:50:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 21:50:23.156548 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a704ed-01ce-4454-97ac-eb150b8595ba None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:23.218052 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 176/360] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:50:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 21:50:23.218872 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411c30dc-c8ec-44c9-84ac-273d9b906019 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:23.292881 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a704ed-01ce-4454-97ac-eb150b8595ba None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:23.300153 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:23.335928 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc711cd-880e-4703-9c1f-63ddca87100c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:23.336612 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc711cd-880e-4703-9c1f-63ddca87100c None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:23.336894 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc711cd-880e-4703-9c1f-63ddca87100c None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:23.337169 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc711cd-880e-4703-9c1f-63ddca87100c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:23.354747 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc711cd-880e-4703-9c1f-63ddca87100c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:50:23.363346 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411c30dc-c8ec-44c9-84ac-273d9b906019 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:23.365299 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-1bc711cd-880e-4703-9c1f-63ddca87100c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 21:50:23.365299 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:23.365299 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:23.365299 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:23.365299 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:23.365299 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:23.365299 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:23.365299 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:23.365299 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:23.365299 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:23.365299 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:23.365299 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:23.365299 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:50:23.365299 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:50:23.365299 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:50:23.365299 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:50:23.366167 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:50:23.366167 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:50:23.366167 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:50:23.366167 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:50:23.366167 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:23.366167 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:23.366167 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:23.366167 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:23.366167 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:23.366167 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:23.366167 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:23.366167 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:23.366167 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:23.366167 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:23.366167 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:23.366167 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:23.366976 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:23.366976 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:23.366976 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:23.366976 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:23.366976 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:50:23.366976 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:50:23.366976 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:50:23.366976 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:50:23.366976 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:50:23.366976 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:50:23.366976 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 21:50:23.366976 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:50:23.366976 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 185/361] 10.222.0.18 () {56 vars in 1114 bytes} [Sat Aug 6 21:50:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:23.367628 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc711cd-880e-4703-9c1f-63ddca87100c None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:23.409458 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9b5592-0da1-4d3a-a3fa-483b14e2093a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:23.410216 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:23.410999 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9b5592-0da1-4d3a-a3fa-483b14e2093a None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:23.411441 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9b5592-0da1-4d3a-a3fa-483b14e2093a None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:23.411852 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9b5592-0da1-4d3a-a3fa-483b14e2093a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:23.501275 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9b5592-0da1-4d3a-a3fa-483b14e2093a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:23.503279 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9b5592-0da1-4d3a-a3fa-483b14e2093a None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:23.503696 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc711cd-880e-4703-9c1f-63ddca87100c None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:23.553817 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d588fa1-0bdb-4cad-b23f-771979d53b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:23.554943 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d588fa1-0bdb-4cad-b23f-771979d53b3d None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:23.555364 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d588fa1-0bdb-4cad-b23f-771979d53b3d None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:23.555787 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d588fa1-0bdb-4cad-b23f-771979d53b3d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:23.570074 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d588fa1-0bdb-4cad-b23f-771979d53b3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:50:23.576571 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-6d588fa1-0bdb-4cad-b23f-771979d53b3d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 21:50:23.576571 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:23.576571 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:23.576571 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:23.576571 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:23.576571 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:23.576571 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:23.576571 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:23.576571 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:23.576571 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:23.576571 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:23.576571 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:23.576571 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:50:23.576571 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:50:23.576571 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:50:23.576571 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:50:23.577667 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:50:23.577667 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:50:23.577667 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:50:23.577667 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:50:23.577667 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:23.577667 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:23.577667 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:23.577667 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:23.577667 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:23.577667 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:23.577667 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:23.577667 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:23.577667 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:23.577667 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:23.577667 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:23.577667 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:23.578682 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:23.578682 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:23.578682 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:23.578682 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:23.578682 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:50:23.578682 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:50:23.578682 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:50:23.578682 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:50:23.578682 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:50:23.578682 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:50:23.578682 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 21:50:23.578682 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:50:23.581986 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 186/362] 10.222.0.18 () {56 vars in 1099 bytes} [Sat Aug 6 21:50:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:23.582939 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d588fa1-0bdb-4cad-b23f-771979d53b3d None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:23.585125 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:23.643279 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 177/363] 10.222.0.18 () {58 vars in 1282 bytes} [Sat Aug 6 21:50:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 21:50:23.644342 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9b5592-0da1-4d3a-a3fa-483b14e2093a None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:23.701709 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:23.713260 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d588fa1-0bdb-4cad-b23f-771979d53b3d None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:23.758878 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4b198f-f4e9-4666-a22a-f8e4d21e90e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:23.759998 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4b198f-f4e9-4666-a22a-f8e4d21e90e2 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:23.760436 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4b198f-f4e9-4666-a22a-f8e4d21e90e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:23.760968 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4b198f-f4e9-4666-a22a-f8e4d21e90e2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:23.768034 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9b5592-0da1-4d3a-a3fa-483b14e2093a None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:23.774910 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4b198f-f4e9-4666-a22a-f8e4d21e90e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:50:23.781024 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-3b4b198f-f4e9-4666-a22a-f8e4d21e90e2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 21:50:23.781024 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:23.781024 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:23.781024 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:23.781024 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:23.781024 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:23.781024 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:23.781024 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:23.781024 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:23.781024 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:23.781024 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:23.781024 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:23.781024 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:50:23.781024 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:50:23.781024 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:50:23.781024 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:50:23.782099 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:50:23.782099 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:50:23.782099 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:50:23.782099 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:50:23.782099 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:23.782099 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:23.782099 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:23.782099 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:23.782099 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:23.782099 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:23.782099 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:23.782099 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:23.782099 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:23.782099 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:23.782099 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:23.782099 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:23.783233 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:23.783233 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:23.783233 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:23.783233 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:23.783233 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:50:23.783233 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:50:23.783233 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:50:23.783233 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:50:23.783233 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:50:23.783233 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:50:23.783233 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 21:50:23.783233 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:50:23.785455 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 187/364] 10.222.0.18 () {56 vars in 1096 bytes} [Sat Aug 6 21:50:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:23.786421 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4b198f-f4e9-4666-a22a-f8e4d21e90e2 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:23.810802 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c42b34-6eb1-4ef3-b23c-b509799a77d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:23.811564 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c42b34-6eb1-4ef3-b23c-b509799a77d9 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:23.811966 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c42b34-6eb1-4ef3-b23c-b509799a77d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:23.812420 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c42b34-6eb1-4ef3-b23c-b509799a77d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:23.813210 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c42b34-6eb1-4ef3-b23c-b509799a77d9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:23.813613 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c42b34-6eb1-4ef3-b23c-b509799a77d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:23.815325 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c42b34-6eb1-4ef3-b23c-b509799a77d9 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:23.826037 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 178/365] 10.222.0.18 () {56 vars in 1111 bytes} [Sat Aug 6 21:50:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:23.827142 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c42b34-6eb1-4ef3-b23c-b509799a77d9 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:23.852861 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:23.923058 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4b198f-f4e9-4666-a22a-f8e4d21e90e2 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:23.949358 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c42b34-6eb1-4ef3-b23c-b509799a77d9 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:23.969725 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aec651a-9ee9-4b43-af0c-6cee3ddaf1ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:23.970650 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aec651a-9ee9-4b43-af0c-6cee3ddaf1ed None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:23.971216 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aec651a-9ee9-4b43-af0c-6cee3ddaf1ed None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:23.971737 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aec651a-9ee9-4b43-af0c-6cee3ddaf1ed None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:23.972628 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aec651a-9ee9-4b43-af0c-6cee3ddaf1ed None admin] RBAC: Adding query filter params (name=reader) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:23.973088 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aec651a-9ee9-4b43-af0c-6cee3ddaf1ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:23.975081 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aec651a-9ee9-4b43-af0c-6cee3ddaf1ed None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:23.986961 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 188/366] 10.222.0.18 () {56 vars in 1101 bytes} [Sat Aug 6 21:50:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:23.988392 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aec651a-9ee9-4b43-af0c-6cee3ddaf1ed None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:23.996028 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4995a731-f4be-4bce-83fc-7d631c9634d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:23.996850 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4995a731-f4be-4bce-83fc-7d631c9634d3 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:23.997925 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4995a731-f4be-4bce-83fc-7d631c9634d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:23.997925 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4995a731-f4be-4bce-83fc-7d631c9634d3 None admin] PATH_INFO: `/v3/projects/0bfd7af746994807b3d8de6f7292ef02` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:23.999354 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4995a731-f4be-4bce-83fc-7d631c9634d3 None admin] RBAC: Authorizing `identity:get_project(project_id=0bfd7af746994807b3d8de6f7292ef02)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:24.000841 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4995a731-f4be-4bce-83fc-7d631c9634d3 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:24.003396 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 179/367] 10.222.0.18 () {56 vars in 1186 bytes} [Sat Aug 6 21:50:23 2022] GET /identity/v3/projects/0bfd7af746994807b3d8de6f7292ef02 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:24.004186 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4995a731-f4be-4bce-83fc-7d631c9634d3 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:24.013348 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:24.106801 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:24.124051 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aec651a-9ee9-4b43-af0c-6cee3ddaf1ed None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:24.131797 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4995a731-f4be-4bce-83fc-7d631c9634d3 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:24.168128 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7455045c-6c62-4ea9-a212-8b624133ad27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:24.169301 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7455045c-6c62-4ea9-a212-8b624133ad27 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:24.169804 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7455045c-6c62-4ea9-a212-8b624133ad27 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:24.170272 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7455045c-6c62-4ea9-a212-8b624133ad27 None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:24.171133 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7455045c-6c62-4ea9-a212-8b624133ad27 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:24.171630 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7455045c-6c62-4ea9-a212-8b624133ad27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:24.173390 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7455045c-6c62-4ea9-a212-8b624133ad27 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:24.182757 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 189/368] 10.222.0.18 () {56 vars in 1099 bytes} [Sat Aug 6 21:50:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:24.183880 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7455045c-6c62-4ea9-a212-8b624133ad27 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:24.190384 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4b7668-2d65-411e-8357-c9e046345e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:24.191023 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4b7668-2d65-411e-8357-c9e046345e99 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:24.191305 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4b7668-2d65-411e-8357-c9e046345e99 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:24.191582 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4b7668-2d65-411e-8357-c9e046345e99 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:24.287246 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4b7668-2d65-411e-8357-c9e046345e99 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:24.289072 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4b7668-2d65-411e-8357-c9e046345e99 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:24.294686 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:24.330115 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7455045c-6c62-4ea9-a212-8b624133ad27 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:24.394323 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9322bb4-3ca4-4459-8c97-f57631f543f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:24.395337 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9322bb4-3ca4-4459-8c97-f57631f543f4 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:24.395813 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9322bb4-3ca4-4459-8c97-f57631f543f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:24.396279 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9322bb4-3ca4-4459-8c97-f57631f543f4 None admin] PATH_INFO: `/v3/users/f22e360a1bb346aba5fb322329f1608c` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:24.398342 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9322bb4-3ca4-4459-8c97-f57631f543f4 None admin] RBAC: Authorizing `identity:get_user(user_id=f22e360a1bb346aba5fb322329f1608c)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:24.401026 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9322bb4-3ca4-4459-8c97-f57631f543f4 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:24.403011 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 190/369] 10.222.0.18 () {56 vars in 1177 bytes} [Sat Aug 6 21:50:24 2022] GET /identity/v3/users/f22e360a1bb346aba5fb322329f1608c => generated 293 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:24.403783 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9322bb4-3ca4-4459-8c97-f57631f543f4 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:24.418571 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:24.442025 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 180/370] 10.222.0.18 () {58 vars in 1282 bytes} [Sat Aug 6 21:50:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 21:50:24.443194 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4b7668-2d65-411e-8357-c9e046345e99 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:24.545520 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9322bb4-3ca4-4459-8c97-f57631f543f4 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:24.568645 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4b7668-2d65-411e-8357-c9e046345e99 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:24.593671 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46301b74-1796-4643-bc93-2702faabcd6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:24.594301 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46301b74-1796-4643-bc93-2702faabcd6a None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:24.594643 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46301b74-1796-4643-bc93-2702faabcd6a None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:24.594916 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46301b74-1796-4643-bc93-2702faabcd6a None admin] PATH_INFO: `/v3/groups/c415347e96884e65a7ab87b53f4860d2` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:24.597337 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46301b74-1796-4643-bc93-2702faabcd6a None admin] RBAC: Authorizing `identity:get_group(group_id=c415347e96884e65a7ab87b53f4860d2)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:24.599057 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46301b74-1796-4643-bc93-2702faabcd6a None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:24.602614 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 191/371] 10.222.0.18 () {56 vars in 1180 bytes} [Sat Aug 6 21:50:24 2022] GET /identity/v3/groups/c415347e96884e65a7ab87b53f4860d2 => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:24.602765 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46301b74-1796-4643-bc93-2702faabcd6a None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:24.615875 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86066996-41ee-4a20-b43b-e9fce8f109f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:24.617018 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86066996-41ee-4a20-b43b-e9fce8f109f3 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:24.617507 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86066996-41ee-4a20-b43b-e9fce8f109f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:24.617984 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86066996-41ee-4a20-b43b-e9fce8f109f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:24.621207 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:24.677306 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:24.703809 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86066996-41ee-4a20-b43b-e9fce8f109f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:24.706344 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86066996-41ee-4a20-b43b-e9fce8f109f3 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:24.731993 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46301b74-1796-4643-bc93-2702faabcd6a None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:24.776969 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebe1e0d-38ff-4086-9d97-618eaaf6ae44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:24.777998 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe1e0d-38ff-4086-9d97-618eaaf6ae44 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:24.778303 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe1e0d-38ff-4086-9d97-618eaaf6ae44 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:24.778573 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe1e0d-38ff-4086-9d97-618eaaf6ae44 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:24.779441 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebe1e0d-38ff-4086-9d97-618eaaf6ae44 None admin] RBAC: Adding query filter params (role.id=817eabff1ca5470993116be35777000a, user.id=f22e360a1bb346aba5fb322329f1608c, scope.system=all) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:24.779859 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebe1e0d-38ff-4086-9d97-618eaaf6ae44 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:24.781594 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebe1e0d-38ff-4086-9d97-618eaaf6ae44 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:24.799949 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 192/372] 10.222.0.18 () {56 vars in 1308 bytes} [Sat Aug 6 21:50:24 2022] GET /identity/v3/role_assignments?user.id=f22e360a1bb346aba5fb322329f1608c&scope.system=all&role.id=817eabff1ca5470993116be35777000a => generated 510 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:24.800768 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebe1e0d-38ff-4086-9d97-618eaaf6ae44 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:24.842881 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 181/373] 10.222.0.18 () {58 vars in 1282 bytes} [Sat Aug 6 21:50:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 21:50:24.843649 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86066996-41ee-4a20-b43b-e9fce8f109f3 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:24.919615 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:24.944922 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:24.957207 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebe1e0d-38ff-4086-9d97-618eaaf6ae44 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:24.967997 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86066996-41ee-4a20-b43b-e9fce8f109f3 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:24.996071 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f175859f-6a0c-4961-92e6-7f577ccb9ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:24.996739 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f175859f-6a0c-4961-92e6-7f577ccb9ddc None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:24.997016 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f175859f-6a0c-4961-92e6-7f577ccb9ddc None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:24.997284 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f175859f-6a0c-4961-92e6-7f577ccb9ddc None admin] PATH_INFO: `/v3/projects/0bfd7af746994807b3d8de6f7292ef02` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:24.998513 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f175859f-6a0c-4961-92e6-7f577ccb9ddc None admin] RBAC: Authorizing `identity:get_project(project_id=0bfd7af746994807b3d8de6f7292ef02)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:24.999965 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f175859f-6a0c-4961-92e6-7f577ccb9ddc None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:25.001672 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 193/374] 10.222.0.18 () {56 vars in 1186 bytes} [Sat Aug 6 21:50:24 2022] GET /identity/v3/projects/0bfd7af746994807b3d8de6f7292ef02 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:25.002477 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f175859f-6a0c-4961-92e6-7f577ccb9ddc None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:25.007279 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6198ca2-27e9-4f8a-9eae-22cdfc624637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:25.008089 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6198ca2-27e9-4f8a-9eae-22cdfc624637 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:25.008584 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6198ca2-27e9-4f8a-9eae-22cdfc624637 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:25.009017 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6198ca2-27e9-4f8a-9eae-22cdfc624637 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:25.009940 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6198ca2-27e9-4f8a-9eae-22cdfc624637 None admin] RBAC: Adding query filter params (group.id=c415347e96884e65a7ab87b53f4860d2, role.id=56e4a96094e9434cbbafb22579c034f5, scope.project.id=0bfd7af746994807b3d8de6f7292ef02) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:25.010386 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6198ca2-27e9-4f8a-9eae-22cdfc624637 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:25.011983 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6198ca2-27e9-4f8a-9eae-22cdfc624637 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:25.022554 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 182/375] 10.222.0.18 () {56 vars in 1376 bytes} [Sat Aug 6 21:50:24 2022] GET /identity/v3/role_assignments?group.id=c415347e96884e65a7ab87b53f4860d2&scope.project.id=0bfd7af746994807b3d8de6f7292ef02&role.id=56e4a96094e9434cbbafb22579c034f5 => generated 260 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:25.023654 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6198ca2-27e9-4f8a-9eae-22cdfc624637 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:25.177441 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6198ca2-27e9-4f8a-9eae-22cdfc624637 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:25.180542 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f175859f-6a0c-4961-92e6-7f577ccb9ddc None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:25.250758 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5c43c6-7d68-47de-9faf-af1b53872238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:25.251379 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5c43c6-7d68-47de-9faf-af1b53872238 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:25.252234 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5c43c6-7d68-47de-9faf-af1b53872238 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:25.252510 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5c43c6-7d68-47de-9faf-af1b53872238 None admin] PATH_INFO: `/v3/groups/c415347e96884e65a7ab87b53f4860d2` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:25.253944 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5c43c6-7d68-47de-9faf-af1b53872238 None admin] RBAC: Authorizing `identity:get_group(group_id=c415347e96884e65a7ab87b53f4860d2)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:25.255401 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5c43c6-7d68-47de-9faf-af1b53872238 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:25.257904 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1d478b-a9ea-4100-a608-acc0dffe5f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:25.258542 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1d478b-a9ea-4100-a608-acc0dffe5f60 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:25.258815 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1d478b-a9ea-4100-a608-acc0dffe5f60 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:25.259082 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1d478b-a9ea-4100-a608-acc0dffe5f60 None admin] PATH_INFO: `/v3/projects/c285ce4925f04fb5acce87c789a71e83` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:25.260200 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 183/376] 10.222.0.18 () {56 vars in 1180 bytes} [Sat Aug 6 21:50:25 2022] GET /identity/v3/groups/c415347e96884e65a7ab87b53f4860d2 => generated 226 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:25.260872 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1d478b-a9ea-4100-a608-acc0dffe5f60 None admin] RBAC: Authorizing `identity:get_project(project_id=c285ce4925f04fb5acce87c789a71e83)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:25.262344 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1d478b-a9ea-4100-a608-acc0dffe5f60 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:25.264314 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 194/377] 10.222.0.18 () {56 vars in 1186 bytes} [Sat Aug 6 21:50:25 2022] GET /identity/v3/projects/c285ce4925f04fb5acce87c789a71e83 => generated 344 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:25.266260 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5c43c6-7d68-47de-9faf-af1b53872238 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:25.270195 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1d478b-a9ea-4100-a608-acc0dffe5f60 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:25.285807 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:25.300797 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:25.412977 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5c43c6-7d68-47de-9faf-af1b53872238 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:25.416625 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1d478b-a9ea-4100-a608-acc0dffe5f60 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:25.453571 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07544a84-6277-4d58-b073-71ed45f37537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:25.454106 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07544a84-6277-4d58-b073-71ed45f37537 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:25.454384 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07544a84-6277-4d58-b073-71ed45f37537 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:25.454658 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07544a84-6277-4d58-b073-71ed45f37537 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:25.455423 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07544a84-6277-4d58-b073-71ed45f37537 None admin] RBAC: Adding query filter params (group.id=c415347e96884e65a7ab87b53f4860d2, role.id=93c058ac49e242fd8517c80c552dea2d, scope.project.id=0bfd7af746994807b3d8de6f7292ef02) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:25.455810 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07544a84-6277-4d58-b073-71ed45f37537 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:25.456251 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1535c33c-b9cc-427f-976c-cdd55598ce64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:25.457077 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535c33c-b9cc-427f-976c-cdd55598ce64 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:25.457295 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07544a84-6277-4d58-b073-71ed45f37537 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:25.457640 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535c33c-b9cc-427f-976c-cdd55598ce64 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:25.458041 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535c33c-b9cc-427f-976c-cdd55598ce64 None admin] PATH_INFO: `/v3/groups/bf403365321e46f48e3808469483a22e` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:25.463394 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 184/378] 10.222.0.18 () {56 vars in 1376 bytes} [Sat Aug 6 21:50:25 2022] GET /identity/v3/role_assignments?group.id=c415347e96884e65a7ab87b53f4860d2&scope.project.id=0bfd7af746994807b3d8de6f7292ef02&role.id=93c058ac49e242fd8517c80c552dea2d => generated 260 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:50:25.463739 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1535c33c-b9cc-427f-976c-cdd55598ce64 None admin] RBAC: Authorizing `identity:get_group(group_id=bf403365321e46f48e3808469483a22e)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:25.464638 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b28e0d-2da4-4684-b4fc-a291f14206bc None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:25.464925 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b28e0d-2da4-4684-b4fc-a291f14206bc None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:25.465197 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b28e0d-2da4-4684-b4fc-a291f14206bc None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:25.465855 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1535c33c-b9cc-427f-976c-cdd55598ce64 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:25.466659 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 185/379] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:50:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:50:25.467738 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23e6283-80e8-4860-800c-9ce5ccf9c6ac None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:25.468009 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23e6283-80e8-4860-800c-9ce5ccf9c6ac None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:25.468359 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23e6283-80e8-4860-800c-9ce5ccf9c6ac None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:25.468519 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 195/380] 10.222.0.18 () {56 vars in 1180 bytes} [Sat Aug 6 21:50:25 2022] GET /identity/v3/groups/bf403365321e46f48e3808469483a22e => generated 229 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:25.472827 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43d97f3-13ef-4a1e-9986-e08b2b7b15eb None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:25.473332 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43d97f3-13ef-4a1e-9986-e08b2b7b15eb None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:25.473748 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43d97f3-13ef-4a1e-9986-e08b2b7b15eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:25.474266 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 186/381] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:50:25 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:50:25.479969 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ec0bef-d42e-4023-8e8a-1737a79872a6 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:25.480249 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ec0bef-d42e-4023-8e8a-1737a79872a6 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:25.480563 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ec0bef-d42e-4023-8e8a-1737a79872a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:25.494073 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-d43d97f3-13ef-4a1e-9986-e08b2b7b15eb None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:50:25.495554 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-d43d97f3-13ef-4a1e-9986-e08b2b7b15eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:50:25.504116 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-55ec0bef-d42e-4023-8e8a-1737a79872a6 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:50:25.505408 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-55ec0bef-d42e-4023-8e8a-1737a79872a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:50:25.572068 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:25.573974 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 196/382] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:50:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 21:50:25.580316 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43d97f3-13ef-4a1e-9986-e08b2b7b15eb None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:25.600922 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 187/383] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:50:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 21:50:25.605841 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ec0bef-d42e-4023-8e8a-1737a79872a6 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:25.615394 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:25.699262 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43d97f3-13ef-4a1e-9986-e08b2b7b15eb None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:25.751252 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d968a9-09f4-4626-ba16-6d036b96c9f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:25.751933 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d968a9-09f4-4626-ba16-6d036b96c9f1 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:25.752287 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d968a9-09f4-4626-ba16-6d036b96c9f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:25.752674 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d968a9-09f4-4626-ba16-6d036b96c9f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:25.753427 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d968a9-09f4-4626-ba16-6d036b96c9f1 None admin] RBAC: Adding query filter params (group.id=bf403365321e46f48e3808469483a22e, role.id=4b8dcf0491fd4072acc69fd894bf77d0, scope.project.id=c285ce4925f04fb5acce87c789a71e83) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:25.753740 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d968a9-09f4-4626-ba16-6d036b96c9f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:25.753847 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ec0bef-d42e-4023-8e8a-1737a79872a6 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:25.755176 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d968a9-09f4-4626-ba16-6d036b96c9f1 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:25.761427 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 197/384] 10.222.0.18 () {56 vars in 1376 bytes} [Sat Aug 6 21:50:25 2022] GET /identity/v3/role_assignments?group.id=bf403365321e46f48e3808469483a22e&scope.project.id=c285ce4925f04fb5acce87c789a71e83&role.id=4b8dcf0491fd4072acc69fd894bf77d0 => generated 260 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:25.762409 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d968a9-09f4-4626-ba16-6d036b96c9f1 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:25.791959 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1322016e-9d66-400f-ad6b-ab42128a2aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:25.792905 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1322016e-9d66-400f-ad6b-ab42128a2aa0 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:25.793380 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1322016e-9d66-400f-ad6b-ab42128a2aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:25.793803 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1322016e-9d66-400f-ad6b-ab42128a2aa0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:25.806532 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1322016e-9d66-400f-ad6b-ab42128a2aa0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:50:25.811899 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-1322016e-9d66-400f-ad6b-ab42128a2aa0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 21:50:25.811899 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:25.811899 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:25.811899 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:25.811899 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:25.811899 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:25.811899 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:25.811899 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:25.811899 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:25.811899 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:25.811899 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:25.811899 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:25.811899 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:50:25.811899 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:50:25.811899 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:50:25.811899 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:50:25.813230 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:50:25.813230 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:50:25.813230 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:50:25.813230 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:50:25.813230 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:25.813230 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:25.813230 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:25.813230 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:25.813230 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:25.813230 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:25.813230 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:25.813230 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:25.813230 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:25.813230 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:25.813230 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:25.813230 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:25.814281 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:25.814281 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:25.814281 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:25.814281 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:25.814281 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:50:25.814281 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:50:25.814281 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:50:25.814281 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:50:25.814281 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:50:25.814281 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:50:25.814281 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 21:50:25.814281 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:50:25.817370 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 188/385] 10.222.0.18 () {56 vars in 1114 bytes} [Sat Aug 6 21:50:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 21:50:25.835970 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1322016e-9d66-400f-ad6b-ab42128a2aa0 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:25.843696 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:25.889977 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d968a9-09f4-4626-ba16-6d036b96c9f1 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:25.930275 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d97e2b-8be8-4c35-bff1-949374291a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:25.931206 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d97e2b-8be8-4c35-bff1-949374291a2d None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:25.931778 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d97e2b-8be8-4c35-bff1-949374291a2d None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:25.932208 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d97e2b-8be8-4c35-bff1-949374291a2d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:25.960337 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d97e2b-8be8-4c35-bff1-949374291a2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:50:25.966656 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:25.972832 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-05d97e2b-8be8-4c35-bff1-949374291a2d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 21:50:25.972832 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:25.972832 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:25.972832 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:25.972832 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:25.972832 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:25.972832 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:25.972832 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:25.972832 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:25.972832 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:25.972832 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:25.972832 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:25.972832 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:50:25.972832 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:50:25.972832 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:50:25.972832 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:50:25.974019 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:50:25.974019 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:50:25.974019 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:50:25.974019 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:50:25.974019 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:25.974019 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:25.974019 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:25.974019 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:25.974019 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:25.974019 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:25.974019 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:25.974019 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:25.974019 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:25.974019 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:25.974019 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:25.974019 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:25.975017 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:25.975017 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:25.975017 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:25.975017 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:25.975017 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:50:25.975017 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:50:25.975017 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:50:25.975017 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:50:25.975017 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:50:25.975017 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:50:25.975017 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 21:50:25.975017 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:50:25.978327 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 198/386] 10.222.0.18 () {56 vars in 1099 bytes} [Sat Aug 6 21:50:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:25.987394 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d97e2b-8be8-4c35-bff1-949374291a2d None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:26.031870 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1322016e-9d66-400f-ad6b-ab42128a2aa0 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:26.087346 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25baa14e-441d-4f65-a535-b049a8a02c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:26.088236 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25baa14e-441d-4f65-a535-b049a8a02c5b None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:26.088788 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25baa14e-441d-4f65-a535-b049a8a02c5b None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:26.089211 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25baa14e-441d-4f65-a535-b049a8a02c5b None admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:26.090539 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25baa14e-441d-4f65-a535-b049a8a02c5b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:26.090982 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25baa14e-441d-4f65-a535-b049a8a02c5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:26.092545 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25baa14e-441d-4f65-a535-b049a8a02c5b None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:26.108554 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 189/387] 10.222.0.18 () {56 vars in 1111 bytes} [Sat Aug 6 21:50:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:50:26.110209 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25baa14e-441d-4f65-a535-b049a8a02c5b None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:26.184609 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d97e2b-8be8-4c35-bff1-949374291a2d None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:26.227528 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:26.231897 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25baa14e-441d-4f65-a535-b049a8a02c5b None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:26.252389 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df09acd-62d6-4322-8f33-0efdf012e553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:26.253460 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df09acd-62d6-4322-8f33-0efdf012e553 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:26.253935 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df09acd-62d6-4322-8f33-0efdf012e553 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:26.254442 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df09acd-62d6-4322-8f33-0efdf012e553 None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:26.255295 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df09acd-62d6-4322-8f33-0efdf012e553 None admin] RBAC: Adding query filter params (name=member) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:26.255815 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df09acd-62d6-4322-8f33-0efdf012e553 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:26.257469 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df09acd-62d6-4322-8f33-0efdf012e553 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:26.264590 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:26.276167 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 199/388] 10.222.0.18 () {56 vars in 1101 bytes} [Sat Aug 6 21:50:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:26.276309 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8269c76b-b6a7-41f1-9334-27bdcdcfec52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:26.278197 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8269c76b-b6a7-41f1-9334-27bdcdcfec52 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:26.278197 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8269c76b-b6a7-41f1-9334-27bdcdcfec52 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:26.278197 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8269c76b-b6a7-41f1-9334-27bdcdcfec52 None admin] PATH_INFO: `/v3/groups/c415347e96884e65a7ab87b53f4860d2` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:26.278845 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8269c76b-b6a7-41f1-9334-27bdcdcfec52 None admin] RBAC: Authorizing `identity:get_group(group_id=c415347e96884e65a7ab87b53f4860d2)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:26.280301 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8269c76b-b6a7-41f1-9334-27bdcdcfec52 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:26.281752 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df09acd-62d6-4322-8f33-0efdf012e553 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:26.284572 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 190/389] 10.222.0.18 () {56 vars in 1180 bytes} [Sat Aug 6 21:50:26 2022] GET /identity/v3/groups/c415347e96884e65a7ab87b53f4860d2 => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:50:26.289043 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8269c76b-b6a7-41f1-9334-27bdcdcfec52 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:26.421073 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8269c76b-b6a7-41f1-9334-27bdcdcfec52 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:26.460346 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fba7330-b723-4148-8b98-6a3f3567c38e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:26.461258 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fba7330-b723-4148-8b98-6a3f3567c38e None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:26.461540 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fba7330-b723-4148-8b98-6a3f3567c38e None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:26.461876 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fba7330-b723-4148-8b98-6a3f3567c38e None admin] PATH_INFO: `/v3/projects/edee8005e8714a9db1c10cfec12857b6` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:26.463415 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fba7330-b723-4148-8b98-6a3f3567c38e None admin] RBAC: Authorizing `identity:get_project(project_id=edee8005e8714a9db1c10cfec12857b6)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:26.464896 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fba7330-b723-4148-8b98-6a3f3567c38e None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:26.467170 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 191/390] 10.222.0.18 () {56 vars in 1186 bytes} [Sat Aug 6 21:50:26 2022] GET /identity/v3/projects/edee8005e8714a9db1c10cfec12857b6 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:50:26.474327 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fba7330-b723-4148-8b98-6a3f3567c38e None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:26.482478 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df09acd-62d6-4322-8f33-0efdf012e553 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:26.522986 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b4f585-e98a-45b1-9656-be5e8213da15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:26.524082 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b4f585-e98a-45b1-9656-be5e8213da15 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:26.524592 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b4f585-e98a-45b1-9656-be5e8213da15 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:26.525077 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b4f585-e98a-45b1-9656-be5e8213da15 None admin] PATH_INFO: `/v3/groups/c415347e96884e65a7ab87b53f4860d2` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:26.526429 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b4f585-e98a-45b1-9656-be5e8213da15 None admin] RBAC: Authorizing `identity:get_group(group_id=c415347e96884e65a7ab87b53f4860d2)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:26.528348 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b4f585-e98a-45b1-9656-be5e8213da15 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:26.530557 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 200/391] 10.222.0.18 () {56 vars in 1180 bytes} [Sat Aug 6 21:50:26 2022] GET /identity/v3/groups/c415347e96884e65a7ab87b53f4860d2 => generated 226 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:50:26.535629 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b4f585-e98a-45b1-9656-be5e8213da15 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:26.545718 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:26.592355 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fba7330-b723-4148-8b98-6a3f3567c38e None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:26.603519 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:26.631062 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14acc45d-c7ed-4d40-b00f-10e69aeb9a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:26.631735 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14acc45d-c7ed-4d40-b00f-10e69aeb9a11 None admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:26.632022 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14acc45d-c7ed-4d40-b00f-10e69aeb9a11 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:26.632311 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14acc45d-c7ed-4d40-b00f-10e69aeb9a11 None admin] PATH_INFO: `/v3/projects/edee8005e8714a9db1c10cfec12857b6/groups/c415347e96884e65a7ab87b53f4860d2/roles/93c058ac49e242fd8517c80c552dea2d` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:26.635007 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14acc45d-c7ed-4d40-b00f-10e69aeb9a11 None admin] RBAC: Authorizing `identity:create_grant(project_id=edee8005e8714a9db1c10cfec12857b6, group_id=c415347e96884e65a7ab87b53f4860d2, role_id=93c058ac49e242fd8517c80c552dea2d)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:26.636499 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14acc45d-c7ed-4d40-b00f-10e69aeb9a11 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:26.659905 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b4f585-e98a-45b1-9656-be5e8213da15 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:26.696820 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9557e1-6cc4-4289-8571-59fce364402f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:26.697799 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9557e1-6cc4-4289-8571-59fce364402f None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:26.698288 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9557e1-6cc4-4289-8571-59fce364402f None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:26.698746 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9557e1-6cc4-4289-8571-59fce364402f None admin] PATH_INFO: `/v3/projects/edee8005e8714a9db1c10cfec12857b6` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:26.700128 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9557e1-6cc4-4289-8571-59fce364402f None admin] RBAC: Authorizing `identity:get_project(project_id=edee8005e8714a9db1c10cfec12857b6)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:26.701833 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9557e1-6cc4-4289-8571-59fce364402f None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:26.704029 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 201/392] 10.222.0.18 () {56 vars in 1186 bytes} [Sat Aug 6 21:50:26 2022] GET /identity/v3/projects/edee8005e8714a9db1c10cfec12857b6 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:50:26.709263 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9557e1-6cc4-4289-8571-59fce364402f None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:26.741723 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 192/393] 10.222.0.18 () {58 vars in 1442 bytes} [Sat Aug 6 21:50:26 2022] PUT /identity/v3/projects/edee8005e8714a9db1c10cfec12857b6/groups/c415347e96884e65a7ab87b53f4860d2/roles/93c058ac49e242fd8517c80c552dea2d => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 21:50:26.826333 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9557e1-6cc4-4289-8571-59fce364402f None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:26.852773 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:26.865701 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0690e1c-848a-4375-9ce3-aa8c6bf52f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:26.866798 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0690e1c-848a-4375-9ce3-aa8c6bf52f17 None admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:26.867270 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0690e1c-848a-4375-9ce3-aa8c6bf52f17 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:26.867747 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0690e1c-848a-4375-9ce3-aa8c6bf52f17 None admin] PATH_INFO: `/v3/projects/edee8005e8714a9db1c10cfec12857b6/groups/c415347e96884e65a7ab87b53f4860d2/roles/56e4a96094e9434cbbafb22579c034f5` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:26.869976 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0690e1c-848a-4375-9ce3-aa8c6bf52f17 None admin] RBAC: Authorizing `identity:create_grant(project_id=edee8005e8714a9db1c10cfec12857b6, group_id=c415347e96884e65a7ab87b53f4860d2, role_id=56e4a96094e9434cbbafb22579c034f5)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:26.871616 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0690e1c-848a-4375-9ce3-aa8c6bf52f17 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:26.951927 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 202/394] 10.222.0.18 () {58 vars in 1442 bytes} [Sat Aug 6 21:50:26 2022] PUT /identity/v3/projects/edee8005e8714a9db1c10cfec12857b6/groups/c415347e96884e65a7ab87b53f4860d2/roles/56e4a96094e9434cbbafb22579c034f5 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 21:50:27.741368 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4997813a-37e1-4b3c-938a-46ec0ecac03f None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:27.741872 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4997813a-37e1-4b3c-938a-46ec0ecac03f None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:27.742155 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4997813a-37e1-4b3c-938a-46ec0ecac03f None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:27.743507 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 193/395] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:50:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:50:27.748868 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c61e7-3510-42d5-94ab-440ddbb88391 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:27.749314 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c61e7-3510-42d5-94ab-440ddbb88391 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:27.749491 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c61e7-3510-42d5-94ab-440ddbb88391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:27.773878 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-845c61e7-3510-42d5-94ab-440ddbb88391 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:50:27.775091 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-845c61e7-3510-42d5-94ab-440ddbb88391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:50:27.855955 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 203/396] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:50:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 21:50:27.900643 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4997813a-37e1-4b3c-938a-46ec0ecac03f None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:28.034937 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4997813a-37e1-4b3c-938a-46ec0ecac03f None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:28.063301 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:28.075371 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdd78e9-d609-4720-a757-7da3243ce8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:28.076010 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd78e9-d609-4720-a757-7da3243ce8af None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:28.076293 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd78e9-d609-4720-a757-7da3243ce8af None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:28.076604 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd78e9-d609-4720-a757-7da3243ce8af None admin] PATH_INFO: `/v3/roles/member` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:28.088812 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdd78e9-d609-4720-a757-7da3243ce8af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:50:28.094102 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-5cdd78e9-d609-4720-a757-7da3243ce8af None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 21:50:28.094102 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:28.094102 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:28.094102 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:28.094102 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:28.094102 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:28.094102 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:28.094102 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:28.094102 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:28.094102 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:28.094102 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:28.094102 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:28.094102 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:50:28.094102 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:50:28.094102 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:50:28.094102 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:50:28.097154 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:50:28.097154 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:50:28.097154 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:50:28.097154 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:50:28.097154 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:28.097154 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:28.097154 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:28.097154 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:28.097154 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:28.097154 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:28.097154 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:28.097154 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:28.097154 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:28.097154 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:28.097154 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:28.097154 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:28.098053 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:28.098053 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:28.098053 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:28.098053 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:28.098053 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:50:28.098053 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:50:28.098053 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:50:28.098053 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:50:28.098053 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:50:28.098053 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:50:28.098053 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 21:50:28.098053 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:50:28.098053 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 194/397] 10.222.0.18 () {56 vars in 1099 bytes} [Sat Aug 6 21:50:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:28.105073 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845c61e7-3510-42d5-94ab-440ddbb88391 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:28.263866 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e92560-7341-4332-93dd-f47b1402ca14 None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:28.264257 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e92560-7341-4332-93dd-f47b1402ca14 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:28.264450 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e92560-7341-4332-93dd-f47b1402ca14 None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:28.266607 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 195/398] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:50:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:50:28.272346 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a34cb3-4b26-4b80-9a2b-57399a35c79a None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:28.272664 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a34cb3-4b26-4b80-9a2b-57399a35c79a None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:28.272933 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a34cb3-4b26-4b80-9a2b-57399a35c79a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:28.275800 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845c61e7-3510-42d5-94ab-440ddbb88391 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:28.294225 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:28.313315 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-58a34cb3-4b26-4b80-9a2b-57399a35c79a None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:50:28.317412 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-58a34cb3-4b26-4b80-9a2b-57399a35c79a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:50:28.337505 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4f5be6-3606-45ad-ae62-4b07a8d488e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:28.338656 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4f5be6-3606-45ad-ae62-4b07a8d488e2 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:28.340603 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4f5be6-3606-45ad-ae62-4b07a8d488e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:28.341587 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4f5be6-3606-45ad-ae62-4b07a8d488e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:28.342637 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4f5be6-3606-45ad-ae62-4b07a8d488e2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:28.343319 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4f5be6-3606-45ad-ae62-4b07a8d488e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:28.347157 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4f5be6-3606-45ad-ae62-4b07a8d488e2 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:28.371859 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 204/399] 10.222.0.18 () {56 vars in 1101 bytes} [Sat Aug 6 21:50:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:28.379920 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4f5be6-3606-45ad-ae62-4b07a8d488e2 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:28.448493 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 196/400] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:50:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 21:50:28.450116 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592b0164-a0bd-46b7-8757-965105cfd561 None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:28.450598 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592b0164-a0bd-46b7-8757-965105cfd561 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:28.451088 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592b0164-a0bd-46b7-8757-965105cfd561 None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:28.452565 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 197/401] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:50:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:50:28.459990 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed386ea-c888-4e8a-aebd-5a1ec57783b6 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:28.460578 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed386ea-c888-4e8a-aebd-5a1ec57783b6 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:28.461048 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed386ea-c888-4e8a-aebd-5a1ec57783b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:28.488060 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-3ed386ea-c888-4e8a-aebd-5a1ec57783b6 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:50:28.489717 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-3ed386ea-c888-4e8a-aebd-5a1ec57783b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:50:28.537904 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4f5be6-3606-45ad-ae62-4b07a8d488e2 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:28.560867 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 198/402] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:50:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 21:50:28.561663 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed386ea-c888-4e8a-aebd-5a1ec57783b6 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:28.596204 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6c72d7-dd94-4717-9a8b-7288ae85c27c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:28.596874 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6c72d7-dd94-4717-9a8b-7288ae85c27c None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:28.597156 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6c72d7-dd94-4717-9a8b-7288ae85c27c None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:28.597436 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6c72d7-dd94-4717-9a8b-7288ae85c27c None admin] PATH_INFO: `/v3/groups/c415347e96884e65a7ab87b53f4860d2` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:28.598870 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6c72d7-dd94-4717-9a8b-7288ae85c27c None admin] RBAC: Authorizing `identity:get_group(group_id=c415347e96884e65a7ab87b53f4860d2)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:28.601443 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6c72d7-dd94-4717-9a8b-7288ae85c27c None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:28.605738 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 205/403] 10.222.0.18 () {56 vars in 1180 bytes} [Sat Aug 6 21:50:28 2022] GET /identity/v3/groups/c415347e96884e65a7ab87b53f4860d2 => generated 226 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:28.606567 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6c72d7-dd94-4717-9a8b-7288ae85c27c None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:28.665338 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:28.674996 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:28.698033 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed386ea-c888-4e8a-aebd-5a1ec57783b6 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:28.737717 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3a9778-45e5-4538-bd61-5ebfb2af8f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:28.739667 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3a9778-45e5-4538-bd61-5ebfb2af8f49 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:28.739884 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3a9778-45e5-4538-bd61-5ebfb2af8f49 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:28.740153 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3a9778-45e5-4538-bd61-5ebfb2af8f49 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:28.743062 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6c72d7-dd94-4717-9a8b-7288ae85c27c None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:28.751846 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3a9778-45e5-4538-bd61-5ebfb2af8f49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:50:28.756999 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-0e3a9778-45e5-4538-bd61-5ebfb2af8f49 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 21:50:28.756999 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:28.756999 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:28.756999 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:28.756999 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:28.756999 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:28.756999 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:28.756999 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:28.756999 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:28.756999 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:28.756999 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:28.756999 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:28.756999 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:50:28.756999 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:50:28.756999 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:50:28.756999 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:50:28.757874 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:50:28.757874 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:50:28.757874 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:50:28.757874 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:50:28.757874 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:28.757874 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:28.757874 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:28.757874 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:28.757874 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:28.757874 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:28.757874 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:28.757874 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:28.757874 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:28.757874 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:28.757874 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:28.757874 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:28.758676 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:28.758676 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:28.758676 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:28.758676 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:28.758676 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:50:28.758676 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:50:28.758676 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:50:28.758676 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:50:28.758676 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:50:28.758676 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:50:28.758676 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 21:50:28.758676 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:50:28.758676 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 199/404] 10.222.0.18 () {56 vars in 1096 bytes} [Sat Aug 6 21:50:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:28.759333 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3a9778-45e5-4538-bd61-5ebfb2af8f49 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:28.783028 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccce6d4-75be-4d15-8f6e-806042c46cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:28.784264 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccce6d4-75be-4d15-8f6e-806042c46cb3 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:28.784693 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccce6d4-75be-4d15-8f6e-806042c46cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:28.785097 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccce6d4-75be-4d15-8f6e-806042c46cb3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:28.795699 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccce6d4-75be-4d15-8f6e-806042c46cb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:50:28.802493 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-0ccce6d4-75be-4d15-8f6e-806042c46cb3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 21:50:28.802493 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:28.802493 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:28.802493 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:28.802493 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:28.802493 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:28.802493 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:28.802493 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:28.802493 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:28.802493 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:28.802493 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:28.802493 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:28.802493 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:50:28.802493 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:50:28.802493 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:50:28.802493 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:50:28.803407 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:50:28.803407 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:50:28.803407 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:50:28.803407 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:50:28.803407 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:28.803407 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:28.803407 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:28.803407 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:28.803407 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:28.803407 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:28.803407 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:28.803407 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:28.803407 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:28.803407 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:28.803407 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:28.803407 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:28.804207 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:28.804207 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:28.804207 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:28.804207 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:28.804207 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:50:28.804207 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:50:28.804207 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:50:28.804207 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:50:28.804207 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:50:28.804207 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:50:28.804207 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 21:50:28.804207 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:50:28.807533 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 206/405] 10.222.0.18 () {56 vars in 1114 bytes} [Sat Aug 6 21:50:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:28.808547 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccce6d4-75be-4d15-8f6e-806042c46cb3 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:28.877508 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3a9778-45e5-4538-bd61-5ebfb2af8f49 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:28.879791 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:28.912058 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccce6d4-75be-4d15-8f6e-806042c46cb3 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:28.914308 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:28.916995 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3dcacb-cd18-4daa-8ac7-5456768c8285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:28.917680 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3dcacb-cd18-4daa-8ac7-5456768c8285 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:28.917954 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3dcacb-cd18-4daa-8ac7-5456768c8285 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:28.918273 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3dcacb-cd18-4daa-8ac7-5456768c8285 None admin] PATH_INFO: `/v3/projects/0bfd7af746994807b3d8de6f7292ef02` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:28.919456 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3dcacb-cd18-4daa-8ac7-5456768c8285 None admin] RBAC: Authorizing `identity:get_project(project_id=0bfd7af746994807b3d8de6f7292ef02)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:28.920938 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3dcacb-cd18-4daa-8ac7-5456768c8285 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:28.928234 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 200/406] 10.222.0.18 () {56 vars in 1186 bytes} [Sat Aug 6 21:50:28 2022] GET /identity/v3/projects/0bfd7af746994807b3d8de6f7292ef02 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:28.928234 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3dcacb-cd18-4daa-8ac7-5456768c8285 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:28.965634 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8168e88-d636-4909-8a52-21ef53c11f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:28.966297 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8168e88-d636-4909-8a52-21ef53c11f1e None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:28.966581 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8168e88-d636-4909-8a52-21ef53c11f1e None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:28.966857 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8168e88-d636-4909-8a52-21ef53c11f1e None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:28.967843 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8168e88-d636-4909-8a52-21ef53c11f1e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:28.968289 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8168e88-d636-4909-8a52-21ef53c11f1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:28.970016 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8168e88-d636-4909-8a52-21ef53c11f1e None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:28.980293 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 207/407] 10.222.0.18 () {56 vars in 1099 bytes} [Sat Aug 6 21:50:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:28.981187 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8168e88-d636-4909-8a52-21ef53c11f1e None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:29.056366 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3dcacb-cd18-4daa-8ac7-5456768c8285 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:29.092122 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8168e88-d636-4909-8a52-21ef53c11f1e None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:29.094080 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe628e8-d78f-45c8-adbc-25bac21b489e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:29.094860 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe628e8-d78f-45c8-adbc-25bac21b489e None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:29.095728 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe628e8-d78f-45c8-adbc-25bac21b489e None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:29.096118 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe628e8-d78f-45c8-adbc-25bac21b489e None admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:29.097083 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe628e8-d78f-45c8-adbc-25bac21b489e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:29.097317 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe628e8-d78f-45c8-adbc-25bac21b489e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:29.098891 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe628e8-d78f-45c8-adbc-25bac21b489e None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:29.108582 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 201/408] 10.222.0.18 () {56 vars in 1111 bytes} [Sat Aug 6 21:50:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:29.114670 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe628e8-d78f-45c8-adbc-25bac21b489e None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:29.132052 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3c2ca0-3c17-4c6d-95cd-270705918747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:29.133088 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3c2ca0-3c17-4c6d-95cd-270705918747 None admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:29.133561 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3c2ca0-3c17-4c6d-95cd-270705918747 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:29.134054 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3c2ca0-3c17-4c6d-95cd-270705918747 None admin] PATH_INFO: `/v3/projects/0bfd7af746994807b3d8de6f7292ef02/groups/c415347e96884e65a7ab87b53f4860d2/roles/56e4a96094e9434cbbafb22579c034f5` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:29.136178 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3c2ca0-3c17-4c6d-95cd-270705918747 None admin] RBAC: Authorizing `identity:create_grant(project_id=0bfd7af746994807b3d8de6f7292ef02, group_id=c415347e96884e65a7ab87b53f4860d2, role_id=56e4a96094e9434cbbafb22579c034f5)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:29.141369 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3c2ca0-3c17-4c6d-95cd-270705918747 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:29.152123 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:29.218529 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe628e8-d78f-45c8-adbc-25bac21b489e None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:29.253918 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405fa563-2f92-4205-8497-ac1c80aa1a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:29.254545 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405fa563-2f92-4205-8497-ac1c80aa1a99 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:29.254822 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405fa563-2f92-4205-8497-ac1c80aa1a99 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:29.255089 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405fa563-2f92-4205-8497-ac1c80aa1a99 None admin] PATH_INFO: `/v3/groups/bf403365321e46f48e3808469483a22e` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:29.256207 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405fa563-2f92-4205-8497-ac1c80aa1a99 None admin] RBAC: Authorizing `identity:get_group(group_id=bf403365321e46f48e3808469483a22e)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:29.257686 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405fa563-2f92-4205-8497-ac1c80aa1a99 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:29.263745 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 202/409] 10.222.0.18 () {56 vars in 1180 bytes} [Sat Aug 6 21:50:29 2022] GET /identity/v3/groups/bf403365321e46f48e3808469483a22e => generated 229 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:29.263745 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405fa563-2f92-4205-8497-ac1c80aa1a99 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:29.272448 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 208/410] 10.222.0.18 () {58 vars in 1442 bytes} [Sat Aug 6 21:50:28 2022] PUT /identity/v3/projects/0bfd7af746994807b3d8de6f7292ef02/groups/c415347e96884e65a7ab87b53f4860d2/roles/56e4a96094e9434cbbafb22579c034f5 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 21:50:29.272448 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e5d37-fa91-46f4-9e6f-4f657a67e30e None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:29.272448 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e5d37-fa91-46f4-9e6f-4f657a67e30e None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:29.272448 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e5d37-fa91-46f4-9e6f-4f657a67e30e None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:29.272988 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 209/411] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:50:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:50:29.277716 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909e5d37-fa91-46f4-9e6f-4f657a67e30e None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:29.320601 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:29.390604 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405fa563-2f92-4205-8497-ac1c80aa1a99 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:29.392430 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:29.432003 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456d6013-3a1b-443a-a0e5-b31fb1c8d504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:29.432661 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456d6013-3a1b-443a-a0e5-b31fb1c8d504 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:29.432942 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456d6013-3a1b-443a-a0e5-b31fb1c8d504 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:29.433217 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456d6013-3a1b-443a-a0e5-b31fb1c8d504 None admin] PATH_INFO: `/v3/groups/c415347e96884e65a7ab87b53f4860d2` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:29.434539 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456d6013-3a1b-443a-a0e5-b31fb1c8d504 None admin] RBAC: Authorizing `identity:get_group(group_id=c415347e96884e65a7ab87b53f4860d2)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:29.436182 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456d6013-3a1b-443a-a0e5-b31fb1c8d504 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:29.437822 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 203/412] 10.222.0.18 () {56 vars in 1180 bytes} [Sat Aug 6 21:50:29 2022] GET /identity/v3/groups/c415347e96884e65a7ab87b53f4860d2 => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:29.438980 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e88bb-de9f-4087-afcf-201ba42a5ffe None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:29.439258 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e88bb-de9f-4087-afcf-201ba42a5ffe None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:29.439526 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e88bb-de9f-4087-afcf-201ba42a5ffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:29.439889 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909e5d37-fa91-46f4-9e6f-4f657a67e30e None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:29.463851 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-eb3e88bb-de9f-4087-afcf-201ba42a5ffe None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:50:29.465198 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-eb3e88bb-de9f-4087-afcf-201ba42a5ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:50:29.487029 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71932a9-572b-4015-b3d9-2d70fdd8a37b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:29.487659 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71932a9-572b-4015-b3d9-2d70fdd8a37b None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:29.487936 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71932a9-572b-4015-b3d9-2d70fdd8a37b None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:29.488201 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71932a9-572b-4015-b3d9-2d70fdd8a37b None admin] PATH_INFO: `/v3/projects/c285ce4925f04fb5acce87c789a71e83` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:29.489475 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71932a9-572b-4015-b3d9-2d70fdd8a37b None admin] RBAC: Authorizing `identity:get_project(project_id=c285ce4925f04fb5acce87c789a71e83)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:29.490963 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71932a9-572b-4015-b3d9-2d70fdd8a37b None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:29.492928 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 210/413] 10.222.0.18 () {56 vars in 1186 bytes} [Sat Aug 6 21:50:29 2022] GET /identity/v3/projects/c285ce4925f04fb5acce87c789a71e83 => generated 344 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:29.493711 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71932a9-572b-4015-b3d9-2d70fdd8a37b None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:29.559267 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 204/414] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:50:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 21:50:29.566805 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3e88bb-de9f-4087-afcf-201ba42a5ffe None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:29.614602 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:29.627122 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71932a9-572b-4015-b3d9-2d70fdd8a37b None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:29.666071 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc6becb-bd34-4448-b940-dbe380679d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:29.666617 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc6becb-bd34-4448-b940-dbe380679d15 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:29.666886 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc6becb-bd34-4448-b940-dbe380679d15 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:29.667160 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc6becb-bd34-4448-b940-dbe380679d15 None admin] PATH_INFO: `/v3/projects/0bfd7af746994807b3d8de6f7292ef02` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:29.668313 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc6becb-bd34-4448-b940-dbe380679d15 None admin] RBAC: Authorizing `identity:get_project(project_id=0bfd7af746994807b3d8de6f7292ef02)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:29.670909 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc6becb-bd34-4448-b940-dbe380679d15 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:29.672899 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 211/415] 10.222.0.18 () {56 vars in 1186 bytes} [Sat Aug 6 21:50:29 2022] GET /identity/v3/projects/0bfd7af746994807b3d8de6f7292ef02 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:29.673754 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc6becb-bd34-4448-b940-dbe380679d15 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:29.695756 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:29.698708 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3e88bb-de9f-4087-afcf-201ba42a5ffe None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:29.736647 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322d09f1-51d2-44c1-91b0-ec6ef98e07d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:29.737295 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d09f1-51d2-44c1-91b0-ec6ef98e07d3 None admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:29.737571 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d09f1-51d2-44c1-91b0-ec6ef98e07d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:29.737839 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d09f1-51d2-44c1-91b0-ec6ef98e07d3 None admin] PATH_INFO: `/v3/projects/c285ce4925f04fb5acce87c789a71e83/groups/bf403365321e46f48e3808469483a22e/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:29.739747 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322d09f1-51d2-44c1-91b0-ec6ef98e07d3 None admin] RBAC: Authorizing `identity:create_grant(project_id=c285ce4925f04fb5acce87c789a71e83, group_id=bf403365321e46f48e3808469483a22e, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:29.741201 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322d09f1-51d2-44c1-91b0-ec6ef98e07d3 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:29.783025 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc6becb-bd34-4448-b940-dbe380679d15 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:29.808211 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 205/416] 10.222.0.18 () {58 vars in 1442 bytes} [Sat Aug 6 21:50:29 2022] PUT /identity/v3/projects/c285ce4925f04fb5acce87c789a71e83/groups/bf403365321e46f48e3808469483a22e/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 21:50:29.809571 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322d09f1-51d2-44c1-91b0-ec6ef98e07d3 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:29.830948 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99937d3-5c8f-40bb-a3fa-5c27ca82aeb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:29.831595 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99937d3-5c8f-40bb-a3fa-5c27ca82aeb6 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:29.831872 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99937d3-5c8f-40bb-a3fa-5c27ca82aeb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:29.832141 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99937d3-5c8f-40bb-a3fa-5c27ca82aeb6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:29.846345 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99937d3-5c8f-40bb-a3fa-5c27ca82aeb6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:50:29.853726 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-b99937d3-5c8f-40bb-a3fa-5c27ca82aeb6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 21:50:29.853726 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:29.853726 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:29.853726 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:29.853726 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:29.853726 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:29.853726 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:29.853726 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:29.853726 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:29.853726 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:29.853726 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:29.853726 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:29.853726 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:50:29.853726 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:50:29.853726 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:50:29.853726 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:50:29.854682 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:50:29.854682 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:50:29.854682 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:50:29.854682 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:50:29.854682 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:29.854682 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:29.854682 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:29.854682 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:29.854682 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:29.854682 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:29.854682 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:29.854682 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:29.854682 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:29.854682 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:29.854682 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:29.854682 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:29.855500 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:29.855500 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:29.855500 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:29.855500 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:29.855500 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:50:29.855500 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:50:29.855500 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:50:29.855500 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:50:29.855500 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:50:29.855500 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:50:29.855500 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 21:50:29.855500 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:50:29.860746 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 212/417] 10.222.0.18 () {56 vars in 1114 bytes} [Sat Aug 6 21:50:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:29.864450 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82c7840-aae2-424e-92b7-748f5e157e6b None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:29.864719 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82c7840-aae2-424e-92b7-748f5e157e6b None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:29.864993 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82c7840-aae2-424e-92b7-748f5e157e6b None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:29.872128 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 213/418] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:50:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:50:29.872128 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82c7840-aae2-424e-92b7-748f5e157e6b None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:29.896840 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:29.961705 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322d09f1-51d2-44c1-91b0-ec6ef98e07d3 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:29.988533 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82c7840-aae2-424e-92b7-748f5e157e6b None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:30.000355 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cea4334-e72a-4175-a5d9-ae04e5d7a8de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:30.001053 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cea4334-e72a-4175-a5d9-ae04e5d7a8de None admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:30.001860 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cea4334-e72a-4175-a5d9-ae04e5d7a8de None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:30.002202 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cea4334-e72a-4175-a5d9-ae04e5d7a8de None admin] PATH_INFO: `/v3/projects/0bfd7af746994807b3d8de6f7292ef02/groups/c415347e96884e65a7ab87b53f4860d2/roles/93c058ac49e242fd8517c80c552dea2d` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:30.005026 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cea4334-e72a-4175-a5d9-ae04e5d7a8de None admin] RBAC: Authorizing `identity:create_grant(project_id=0bfd7af746994807b3d8de6f7292ef02, group_id=c415347e96884e65a7ab87b53f4860d2, role_id=93c058ac49e242fd8517c80c552dea2d)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:30.006596 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cea4334-e72a-4175-a5d9-ae04e5d7a8de None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:30.032235 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068fa19d-63f5-41f4-9168-8185c719cde6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:30.033759 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068fa19d-63f5-41f4-9168-8185c719cde6 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:30.034195 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068fa19d-63f5-41f4-9168-8185c719cde6 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:30.034603 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068fa19d-63f5-41f4-9168-8185c719cde6 None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:30.035453 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068fa19d-63f5-41f4-9168-8185c719cde6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:30.035905 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068fa19d-63f5-41f4-9168-8185c719cde6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:30.037450 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068fa19d-63f5-41f4-9168-8185c719cde6 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:30.046545 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 214/419] 10.222.0.18 () {56 vars in 1111 bytes} [Sat Aug 6 21:50:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:30.047999 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e524dd-db1a-45db-b3c6-6c9fb979cc0b None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:30.050884 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e524dd-db1a-45db-b3c6-6c9fb979cc0b None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:30.050884 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e524dd-db1a-45db-b3c6-6c9fb979cc0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:30.068201 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-80e524dd-db1a-45db-b3c6-6c9fb979cc0b None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:50:30.070003 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-80e524dd-db1a-45db-b3c6-6c9fb979cc0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:50:30.078184 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 206/420] 10.222.0.18 () {58 vars in 1442 bytes} [Sat Aug 6 21:50:29 2022] PUT /identity/v3/projects/0bfd7af746994807b3d8de6f7292ef02/groups/c415347e96884e65a7ab87b53f4860d2/roles/93c058ac49e242fd8517c80c552dea2d => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 21:50:30.086670 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cea4334-e72a-4175-a5d9-ae04e5d7a8de None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:30.142827 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:30.239075 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 215/421] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:50:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 21:50:30.268854 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e524dd-db1a-45db-b3c6-6c9fb979cc0b None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:30.285517 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:30.297461 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cea4334-e72a-4175-a5d9-ae04e5d7a8de None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:30.340260 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c420276-c1b9-40df-b236-fc79bb239d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:30.340914 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c420276-c1b9-40df-b236-fc79bb239d30 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:30.341188 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c420276-c1b9-40df-b236-fc79bb239d30 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:30.341539 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c420276-c1b9-40df-b236-fc79bb239d30 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:30.391055 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e524dd-db1a-45db-b3c6-6c9fb979cc0b None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:30.436151 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d873e175-b23f-4ace-9e2a-a530e5012293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:30.436903 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d873e175-b23f-4ace-9e2a-a530e5012293 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:30.437182 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d873e175-b23f-4ace-9e2a-a530e5012293 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:30.437447 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d873e175-b23f-4ace-9e2a-a530e5012293 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:30.446447 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c420276-c1b9-40df-b236-fc79bb239d30 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:30.447914 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c420276-c1b9-40df-b236-fc79bb239d30 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:30.459600 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d873e175-b23f-4ace-9e2a-a530e5012293 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:50:30.467351 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-d873e175-b23f-4ace-9e2a-a530e5012293 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 21:50:30.467351 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:30.467351 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:30.467351 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:30.467351 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:30.467351 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:30.467351 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:30.467351 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:30.467351 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:30.467351 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:30.467351 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:30.467351 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:30.467351 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:50:30.467351 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:50:30.467351 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:50:30.467351 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:50:30.468328 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:50:30.468328 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:50:30.468328 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:50:30.468328 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:50:30.468328 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:30.468328 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:30.468328 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:30.468328 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:30.468328 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:30.468328 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:30.468328 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:30.468328 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:30.468328 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:30.468328 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:30.468328 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:30.468328 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:30.469148 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:30.469148 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:30.469148 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:30.469148 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:30.469148 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:50:30.469148 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:50:30.469148 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:50:30.469148 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:50:30.469148 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:50:30.469148 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:50:30.469148 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 21:50:30.469148 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:50:30.469148 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 216/422] 10.222.0.18 () {56 vars in 1099 bytes} [Sat Aug 6 21:50:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 21:50:30.471205 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:30.477228 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d873e175-b23f-4ace-9e2a-a530e5012293 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:30.583605 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 207/423] 10.222.0.18 () {58 vars in 1282 bytes} [Sat Aug 6 21:50:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 21:50:30.583902 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:30.590999 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c420276-c1b9-40df-b236-fc79bb239d30 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:30.613721 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d873e175-b23f-4ace-9e2a-a530e5012293 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:30.653120 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e176ff5f-2f43-4855-bdf8-d7d66d05989c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:30.654077 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e176ff5f-2f43-4855-bdf8-d7d66d05989c None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:30.654617 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e176ff5f-2f43-4855-bdf8-d7d66d05989c None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:30.655170 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e176ff5f-2f43-4855-bdf8-d7d66d05989c None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:30.656134 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e176ff5f-2f43-4855-bdf8-d7d66d05989c None admin] RBAC: Adding query filter params (name=member) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:30.656680 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e176ff5f-2f43-4855-bdf8-d7d66d05989c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:30.659620 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e176ff5f-2f43-4855-bdf8-d7d66d05989c None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:30.673828 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 217/424] 10.222.0.18 () {56 vars in 1101 bytes} [Sat Aug 6 21:50:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:50:30.679558 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e176ff5f-2f43-4855-bdf8-d7d66d05989c None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:30.819552 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c420276-c1b9-40df-b236-fc79bb239d30 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:30.825973 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e176ff5f-2f43-4855-bdf8-d7d66d05989c None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:30.858319 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:30.884122 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae7e1ce-7057-46d0-93d2-8105d061eace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:30.885924 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae7e1ce-7057-46d0-93d2-8105d061eace None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:30.886931 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae7e1ce-7057-46d0-93d2-8105d061eace None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:30.887767 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ec4ee2-2719-4a2a-8543-f804490f6be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:30.888266 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae7e1ce-7057-46d0-93d2-8105d061eace None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:30.889662 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ec4ee2-2719-4a2a-8543-f804490f6be6 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:30.890502 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ec4ee2-2719-4a2a-8543-f804490f6be6 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:30.891824 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ec4ee2-2719-4a2a-8543-f804490f6be6 None admin] PATH_INFO: `/v3/projects/edee8005e8714a9db1c10cfec12857b6` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:30.894267 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ec4ee2-2719-4a2a-8543-f804490f6be6 None admin] RBAC: Authorizing `identity:get_project(project_id=edee8005e8714a9db1c10cfec12857b6)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:30.895389 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:30.896033 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ec4ee2-2719-4a2a-8543-f804490f6be6 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:30.898772 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 208/425] 10.222.0.18 () {56 vars in 1186 bytes} [Sat Aug 6 21:50:30 2022] GET /identity/v3/projects/edee8005e8714a9db1c10cfec12857b6 => generated 298 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:30.909382 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ec4ee2-2719-4a2a-8543-f804490f6be6 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:30.976158 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae7e1ce-7057-46d0-93d2-8105d061eace None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:30.977862 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae7e1ce-7057-46d0-93d2-8105d061eace None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:31.100450 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ec4ee2-2719-4a2a-8543-f804490f6be6 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:31.145523 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f6c8fe-7232-4bbd-8de1-11c196b77a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:31.145712 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 218/426] 10.222.0.18 () {58 vars in 1282 bytes} [Sat Aug 6 21:50:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 06 21:50:31.146480 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f6c8fe-7232-4bbd-8de1-11c196b77a63 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:31.151210 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f6c8fe-7232-4bbd-8de1-11c196b77a63 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:31.151730 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f6c8fe-7232-4bbd-8de1-11c196b77a63 None admin] PATH_INFO: `/v3/groups/c415347e96884e65a7ab87b53f4860d2` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:31.152323 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae7e1ce-7057-46d0-93d2-8105d061eace None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:31.153224 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f6c8fe-7232-4bbd-8de1-11c196b77a63 None admin] RBAC: Authorizing `identity:get_group(group_id=c415347e96884e65a7ab87b53f4860d2)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:31.154836 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f6c8fe-7232-4bbd-8de1-11c196b77a63 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:31.157266 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 209/427] 10.222.0.18 () {56 vars in 1180 bytes} [Sat Aug 6 21:50:30 2022] GET /identity/v3/groups/c415347e96884e65a7ab87b53f4860d2 => generated 226 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:31.165205 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f6c8fe-7232-4bbd-8de1-11c196b77a63 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:31.222386 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:31.225641 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:31.272207 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f6c8fe-7232-4bbd-8de1-11c196b77a63 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:31.314849 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6a677d-011b-49f3-ab40-dc3668a122e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:31.316576 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6a677d-011b-49f3-ab40-dc3668a122e8 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:31.318346 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6a677d-011b-49f3-ab40-dc3668a122e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:31.318765 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6a677d-011b-49f3-ab40-dc3668a122e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:31.319637 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6a677d-011b-49f3-ab40-dc3668a122e8 None admin] RBAC: Adding query filter params (group.id=c415347e96884e65a7ab87b53f4860d2, role.id=93c058ac49e242fd8517c80c552dea2d, scope.project.id=edee8005e8714a9db1c10cfec12857b6) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:31.320910 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6a677d-011b-49f3-ab40-dc3668a122e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:31.322490 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6a677d-011b-49f3-ab40-dc3668a122e8 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:31.335573 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 210/428] 10.222.0.18 () {56 vars in 1376 bytes} [Sat Aug 6 21:50:31 2022] GET /identity/v3/role_assignments?group.id=c415347e96884e65a7ab87b53f4860d2&scope.project.id=edee8005e8714a9db1c10cfec12857b6&role.id=93c058ac49e242fd8517c80c552dea2d => generated 611 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:31.366038 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae7e1ce-7057-46d0-93d2-8105d061eace None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:31.424032 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c585d78f-d6b4-4d1e-8f1b-61215e7c6d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:31.425009 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c585d78f-d6b4-4d1e-8f1b-61215e7c6d9e None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:31.425432 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c585d78f-d6b4-4d1e-8f1b-61215e7c6d9e None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:31.425842 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c585d78f-d6b4-4d1e-8f1b-61215e7c6d9e None admin] PATH_INFO: `/v3/projects/edee8005e8714a9db1c10cfec12857b6` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:31.431412 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c585d78f-d6b4-4d1e-8f1b-61215e7c6d9e None admin] RBAC: Authorizing `identity:get_project(project_id=edee8005e8714a9db1c10cfec12857b6)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:31.433197 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c585d78f-d6b4-4d1e-8f1b-61215e7c6d9e None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:31.435477 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 219/429] 10.222.0.18 () {56 vars in 1186 bytes} [Sat Aug 6 21:50:31 2022] GET /identity/v3/projects/edee8005e8714a9db1c10cfec12857b6 => generated 298 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:50:31.442223 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6a677d-011b-49f3-ab40-dc3668a122e8 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:31.554760 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6a677d-011b-49f3-ab40-dc3668a122e8 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:31.577537 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324cc8f3-3610-4b14-a9cb-93879cc51415 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:31.577887 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324cc8f3-3610-4b14-a9cb-93879cc51415 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:31.578087 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324cc8f3-3610-4b14-a9cb-93879cc51415 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:31.579612 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 220/430] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:50:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:50:31.581712 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:31.586797 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa92a58-cf3d-4fea-9ef8-3d389b426f45 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:31.586797 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa92a58-cf3d-4fea-9ef8-3d389b426f45 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:31.586797 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa92a58-cf3d-4fea-9ef8-3d389b426f45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:31.597985 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fbdb93-2312-46ad-b169-e3b1138eb0b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:31.598672 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fbdb93-2312-46ad-b169-e3b1138eb0b2 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:31.598954 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fbdb93-2312-46ad-b169-e3b1138eb0b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:31.599224 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fbdb93-2312-46ad-b169-e3b1138eb0b2 None admin] PATH_INFO: `/v3/groups/c415347e96884e65a7ab87b53f4860d2` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:31.600343 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fbdb93-2312-46ad-b169-e3b1138eb0b2 None admin] RBAC: Authorizing `identity:get_group(group_id=c415347e96884e65a7ab87b53f4860d2)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:31.601844 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fbdb93-2312-46ad-b169-e3b1138eb0b2 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:31.603732 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 211/431] 10.222.0.18 () {56 vars in 1180 bytes} [Sat Aug 6 21:50:31 2022] GET /identity/v3/groups/c415347e96884e65a7ab87b53f4860d2 => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:31.609603 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-9fa92a58-cf3d-4fea-9ef8-3d389b426f45 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:50:31.609759 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fbdb93-2312-46ad-b169-e3b1138eb0b2 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:31.611774 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-9fa92a58-cf3d-4fea-9ef8-3d389b426f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:50:31.695326 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 221/432] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:50:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 21:50:31.719113 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa92a58-cf3d-4fea-9ef8-3d389b426f45 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:31.731891 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fbdb93-2312-46ad-b169-e3b1138eb0b2 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:31.756776 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:31.770214 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f04c23b-c61f-41e5-8e1c-d857118fe4e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:31.772586 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f04c23b-c61f-41e5-8e1c-d857118fe4e6 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:31.772586 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f04c23b-c61f-41e5-8e1c-d857118fe4e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:31.772586 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f04c23b-c61f-41e5-8e1c-d857118fe4e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:31.772586 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f04c23b-c61f-41e5-8e1c-d857118fe4e6 None admin] RBAC: Adding query filter params (group.id=c415347e96884e65a7ab87b53f4860d2, role.id=56e4a96094e9434cbbafb22579c034f5, scope.project.id=edee8005e8714a9db1c10cfec12857b6) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:31.772586 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f04c23b-c61f-41e5-8e1c-d857118fe4e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:31.773789 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f04c23b-c61f-41e5-8e1c-d857118fe4e6 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:31.780229 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 212/433] 10.222.0.18 () {56 vars in 1376 bytes} [Sat Aug 6 21:50:31 2022] GET /identity/v3/role_assignments?group.id=c415347e96884e65a7ab87b53f4860d2&scope.project.id=edee8005e8714a9db1c10cfec12857b6&role.id=56e4a96094e9434cbbafb22579c034f5 => generated 611 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:50:31.847427 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa92a58-cf3d-4fea-9ef8-3d389b426f45 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:31.887476 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175f52ef-3624-41e2-8ffb-b31a053d9809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:31.888126 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175f52ef-3624-41e2-8ffb-b31a053d9809 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:31.888422 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175f52ef-3624-41e2-8ffb-b31a053d9809 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:31.888705 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175f52ef-3624-41e2-8ffb-b31a053d9809 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:31.905812 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175f52ef-3624-41e2-8ffb-b31a053d9809 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:50:31.910723 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-175f52ef-3624-41e2-8ffb-b31a053d9809 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 21:50:31.910723 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:31.910723 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:31.910723 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:31.910723 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:31.910723 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:31.910723 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:31.910723 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:31.910723 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:31.910723 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:31.910723 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:31.910723 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:31.910723 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:50:31.910723 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:50:31.910723 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:50:31.910723 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:50:31.911621 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:50:31.911621 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:50:31.911621 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:50:31.911621 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:50:31.911621 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:31.911621 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:31.911621 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:31.911621 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:31.911621 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:31.911621 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:31.911621 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:31.911621 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:31.911621 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:31.911621 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:31.911621 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:31.911621 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:31.913581 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:31.913581 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:31.913581 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:31.913581 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:31.913581 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:50:31.913581 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:50:31.913581 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:50:31.913581 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:50:31.913581 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:50:31.913581 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:50:31.913581 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 21:50:31.913581 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:50:31.913581 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 222/434] 10.222.0.18 () {56 vars in 1099 bytes} [Sat Aug 6 21:50:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:31.918234 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f04c23b-c61f-41e5-8e1c-d857118fe4e6 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:31.978541 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:32.033802 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f04c23b-c61f-41e5-8e1c-d857118fe4e6 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:32.071610 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c018ac91-6d4e-4e0d-b3de-c6d2879823ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:32.072222 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c018ac91-6d4e-4e0d-b3de-c6d2879823ce None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:32.072534 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c018ac91-6d4e-4e0d-b3de-c6d2879823ce None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:32.072812 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c018ac91-6d4e-4e0d-b3de-c6d2879823ce None admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:32.073488 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c018ac91-6d4e-4e0d-b3de-c6d2879823ce None admin] RBAC: Adding query filter params (name=member) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:32.073792 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c018ac91-6d4e-4e0d-b3de-c6d2879823ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:32.075183 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c018ac91-6d4e-4e0d-b3de-c6d2879823ce None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:32.084295 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 213/435] 10.222.0.18 () {56 vars in 1101 bytes} [Sat Aug 6 21:50:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:32.089873 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175f52ef-3624-41e2-8ffb-b31a053d9809 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:32.137726 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc9e5c-1c2b-442d-b847-03697f36b96c None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:32.138058 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc9e5c-1c2b-442d-b847-03697f36b96c None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:32.138271 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc9e5c-1c2b-442d-b847-03697f36b96c None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:32.139876 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 214/436] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:50:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 21:50:32.145598 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94260917-c9f5-410f-b867-40aaf0bb171c None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:32.145973 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94260917-c9f5-410f-b867-40aaf0bb171c None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:32.146255 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94260917-c9f5-410f-b867-40aaf0bb171c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:32.168861 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-94260917-c9f5-410f-b867-40aaf0bb171c None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:50:32.170083 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-94260917-c9f5-410f-b867-40aaf0bb171c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:50:32.198619 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:32.212485 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175f52ef-3624-41e2-8ffb-b31a053d9809 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:32.272538 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 215/437] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:50:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 21:50:32.276677 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6e5824-3ceb-446c-81e4-94a43e746832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:32.277334 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6e5824-3ceb-446c-81e4-94a43e746832 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:32.277612 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6e5824-3ceb-446c-81e4-94a43e746832 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:32.277878 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6e5824-3ceb-446c-81e4-94a43e746832 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:32.295170 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94260917-c9f5-410f-b867-40aaf0bb171c None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:32.361520 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:32.366474 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6e5824-3ceb-446c-81e4-94a43e746832 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:32.367338 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6e5824-3ceb-446c-81e4-94a43e746832 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:32.434334 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94260917-c9f5-410f-b867-40aaf0bb171c None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:32.481023 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:32.483451 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d3c2c2-0b65-4a40-aacd-a55bd5bc6d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:32.484096 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d3c2c2-0b65-4a40-aacd-a55bd5bc6d45 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:32.484373 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d3c2c2-0b65-4a40-aacd-a55bd5bc6d45 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:32.484675 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d3c2c2-0b65-4a40-aacd-a55bd5bc6d45 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:32.494857 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 223/438] 10.222.0.18 () {58 vars in 1282 bytes} [Sat Aug 6 21:50:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 21:50:32.498420 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d3c2c2-0b65-4a40-aacd-a55bd5bc6d45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:50:32.500111 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6e5824-3ceb-446c-81e4-94a43e746832 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:32.503330 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-61d3c2c2-0b65-4a40-aacd-a55bd5bc6d45 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 21:50:32.503330 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:32.503330 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:32.503330 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:32.503330 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:32.503330 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:32.503330 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:32.503330 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:32.503330 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:32.503330 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:32.503330 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:32.503330 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:32.503330 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:50:32.503330 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:50:32.503330 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:50:32.503330 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:50:32.506258 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:50:32.506258 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:50:32.506258 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:50:32.506258 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:50:32.506258 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:32.506258 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:32.506258 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:32.506258 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:32.506258 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:32.506258 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:32.506258 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:32.506258 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:32.506258 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:32.506258 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:32.506258 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:32.506258 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:32.507107 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:32.507107 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:32.507107 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:32.507107 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:32.507107 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:50:32.507107 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:50:32.507107 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:50:32.507107 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:50:32.507107 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:50:32.507107 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:50:32.507107 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 21:50:32.507107 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:50:32.507107 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 216/439] 10.222.0.18 () {56 vars in 1096 bytes} [Sat Aug 6 21:50:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 21:50:32.510709 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95873ff-db64-40d1-9565-7227ca52795d None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:32.510991 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95873ff-db64-40d1-9565-7227ca52795d None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:32.511261 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95873ff-db64-40d1-9565-7227ca52795d None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:32.512694 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 217/440] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:50:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 21:50:32.513262 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95873ff-db64-40d1-9565-7227ca52795d None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:32.607511 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6e5824-3ceb-446c-81e4-94a43e746832 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:32.615004 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95873ff-db64-40d1-9565-7227ca52795d None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:32.617250 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:32.640101 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d49a276-58bc-4ca6-b48f-d82fac2d90dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:32.640749 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d49a276-58bc-4ca6-b48f-d82fac2d90dd None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:32.641025 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d49a276-58bc-4ca6-b48f-d82fac2d90dd None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:32.641295 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d49a276-58bc-4ca6-b48f-d82fac2d90dd None admin] PATH_INFO: `/v3/projects/0bfd7af746994807b3d8de6f7292ef02` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:32.642994 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d49a276-58bc-4ca6-b48f-d82fac2d90dd None admin] RBAC: Authorizing `identity:get_project(project_id=0bfd7af746994807b3d8de6f7292ef02)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:32.644459 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d49a276-58bc-4ca6-b48f-d82fac2d90dd None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:32.646065 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 224/441] 10.222.0.18 () {56 vars in 1186 bytes} [Sat Aug 6 21:50:32 2022] GET /identity/v3/projects/0bfd7af746994807b3d8de6f7292ef02 => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:50:32.647158 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23730142-5463-45e8-813b-b2856611b48f None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:32.647437 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23730142-5463-45e8-813b-b2856611b48f None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:32.647709 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23730142-5463-45e8-813b-b2856611b48f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:32.648942 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2636281f-5c30-4787-9709-bb92343ac0aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:32.649547 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2636281f-5c30-4787-9709-bb92343ac0aa None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:32.649823 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2636281f-5c30-4787-9709-bb92343ac0aa None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:32.650156 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2636281f-5c30-4787-9709-bb92343ac0aa None admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:32.650761 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2636281f-5c30-4787-9709-bb92343ac0aa None admin] RBAC: Adding query filter params (name=admin) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:32.651063 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2636281f-5c30-4787-9709-bb92343ac0aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:32.652432 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2636281f-5c30-4787-9709-bb92343ac0aa None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:32.662674 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 218/442] 10.222.0.18 () {56 vars in 1099 bytes} [Sat Aug 6 21:50:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:32.663473 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2636281f-5c30-4787-9709-bb92343ac0aa None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:32.665585 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-23730142-5463-45e8-813b-b2856611b48f None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:50:32.666767 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-23730142-5463-45e8-813b-b2856611b48f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:50:32.722183 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:32.732902 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 225/443] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:50:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 21:50:32.734123 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23730142-5463-45e8-813b-b2856611b48f None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:32.763644 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2636281f-5c30-4787-9709-bb92343ac0aa None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:32.795980 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f53556-5343-462f-8d4e-3391203543b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:32.796773 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f53556-5343-462f-8d4e-3391203543b3 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:32.797046 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f53556-5343-462f-8d4e-3391203543b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:32.797314 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f53556-5343-462f-8d4e-3391203543b3 None admin] PATH_INFO: `/v3/groups/c415347e96884e65a7ab87b53f4860d2` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:32.798960 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f53556-5343-462f-8d4e-3391203543b3 None admin] RBAC: Authorizing `identity:get_group(group_id=c415347e96884e65a7ab87b53f4860d2)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:32.800376 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f53556-5343-462f-8d4e-3391203543b3 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:32.801948 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 219/444] 10.222.0.18 () {56 vars in 1180 bytes} [Sat Aug 6 21:50:32 2022] GET /identity/v3/groups/c415347e96884e65a7ab87b53f4860d2 => generated 226 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:32.802726 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f53556-5343-462f-8d4e-3391203543b3 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:32.834569 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23730142-5463-45e8-813b-b2856611b48f None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:32.835831 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:32.867968 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5472df-c786-4292-bed2-dda434b1c45e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:32.868519 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5472df-c786-4292-bed2-dda434b1c45e None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:32.868793 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5472df-c786-4292-bed2-dda434b1c45e None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:32.869062 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5472df-c786-4292-bed2-dda434b1c45e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:32.901552 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f53556-5343-462f-8d4e-3391203543b3 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:32.925436 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:32.934695 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e6075e-53da-43fd-acd0-896b195d1e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:32.935382 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e6075e-53da-43fd-acd0-896b195d1e34 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:32.941617 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5472df-c786-4292-bed2-dda434b1c45e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:32.941747 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e6075e-53da-43fd-acd0-896b195d1e34 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:32.941747 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e6075e-53da-43fd-acd0-896b195d1e34 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:32.941904 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5472df-c786-4292-bed2-dda434b1c45e None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:32.946751 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e6075e-53da-43fd-acd0-896b195d1e34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:50:32.951538 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-c7e6075e-53da-43fd-acd0-896b195d1e34 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 21:50:32.951538 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:32.951538 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:32.951538 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:32.951538 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:32.951538 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:32.951538 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:32.951538 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:32.951538 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:32.951538 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:32.951538 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:32.951538 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:32.951538 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:50:32.951538 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:50:32.951538 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:50:32.951538 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:50:32.952389 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:50:32.952389 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:50:32.952389 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:50:32.952389 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:50:32.952389 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:32.952389 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:32.952389 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:32.952389 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:32.952389 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:32.952389 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:32.952389 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:32.952389 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:32.952389 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:32.952389 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:32.952389 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:32.952389 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:32.953208 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:32.953208 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:32.953208 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:32.953208 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:32.953208 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:50:32.953208 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:50:32.953208 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:50:32.953208 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:50:32.953208 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:50:32.953208 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:50:32.953208 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 21:50:32.953208 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:50:32.953208 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 220/445] 10.222.0.18 () {56 vars in 1114 bytes} [Sat Aug 6 21:50:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:32.954357 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e6075e-53da-43fd-acd0-896b195d1e34 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:33.043105 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 226/446] 10.222.0.18 () {58 vars in 1282 bytes} [Sat Aug 6 21:50:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 21:50:33.043955 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5472df-c786-4292-bed2-dda434b1c45e None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:33.045873 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:33.052942 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e6075e-53da-43fd-acd0-896b195d1e34 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:33.087237 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd7ed3a-2bd9-42c4-8f2c-749d6b66a0be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:33.087828 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd7ed3a-2bd9-42c4-8f2c-749d6b66a0be None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:33.088110 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd7ed3a-2bd9-42c4-8f2c-749d6b66a0be None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:33.088376 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd7ed3a-2bd9-42c4-8f2c-749d6b66a0be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:33.089131 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd7ed3a-2bd9-42c4-8f2c-749d6b66a0be None admin] RBAC: Adding query filter params (group.id=c415347e96884e65a7ab87b53f4860d2, role.id=56e4a96094e9434cbbafb22579c034f5, scope.project.id=0bfd7af746994807b3d8de6f7292ef02) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:33.089429 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd7ed3a-2bd9-42c4-8f2c-749d6b66a0be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:33.090844 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd7ed3a-2bd9-42c4-8f2c-749d6b66a0be None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:33.097334 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 221/447] 10.222.0.18 () {56 vars in 1376 bytes} [Sat Aug 6 21:50:32 2022] GET /identity/v3/role_assignments?group.id=c415347e96884e65a7ab87b53f4860d2&scope.project.id=0bfd7af746994807b3d8de6f7292ef02&role.id=56e4a96094e9434cbbafb22579c034f5 => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:33.098188 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd7ed3a-2bd9-42c4-8f2c-749d6b66a0be None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:33.161354 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:33.170635 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5472df-c786-4292-bed2-dda434b1c45e None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:33.208856 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d3ea99-2435-4d66-a6fb-10df54ba4f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:33.209475 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d3ea99-2435-4d66-a6fb-10df54ba4f89 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:33.209755 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d3ea99-2435-4d66-a6fb-10df54ba4f89 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:33.210055 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d3ea99-2435-4d66-a6fb-10df54ba4f89 None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:33.210741 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d3ea99-2435-4d66-a6fb-10df54ba4f89 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:33.211046 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d3ea99-2435-4d66-a6fb-10df54ba4f89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:33.212493 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d3ea99-2435-4d66-a6fb-10df54ba4f89 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:33.218227 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd7ed3a-2bd9-42c4-8f2c-749d6b66a0be None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:33.223084 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 227/448] 10.222.0.18 () {56 vars in 1111 bytes} [Sat Aug 6 21:50:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:33.228648 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d3ea99-2435-4d66-a6fb-10df54ba4f89 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:33.264007 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afdd847-5fe7-4b3d-ac96-a9fad7030e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:33.265038 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afdd847-5fe7-4b3d-ac96-a9fad7030e22 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:33.265038 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afdd847-5fe7-4b3d-ac96-a9fad7030e22 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:33.265237 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afdd847-5fe7-4b3d-ac96-a9fad7030e22 None admin] PATH_INFO: `/v3/projects/c285ce4925f04fb5acce87c789a71e83` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:33.266304 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afdd847-5fe7-4b3d-ac96-a9fad7030e22 None admin] RBAC: Authorizing `identity:get_project(project_id=c285ce4925f04fb5acce87c789a71e83)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:33.268395 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afdd847-5fe7-4b3d-ac96-a9fad7030e22 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:33.270348 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 222/449] 10.222.0.18 () {56 vars in 1186 bytes} [Sat Aug 6 21:50:33 2022] GET /identity/v3/projects/c285ce4925f04fb5acce87c789a71e83 => generated 344 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:33.275511 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afdd847-5fe7-4b3d-ac96-a9fad7030e22 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:33.310762 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:33.347564 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d3ea99-2435-4d66-a6fb-10df54ba4f89 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:33.396345 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69778522-d2be-4e28-ab45-38ba61d6c663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:33.397025 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69778522-d2be-4e28-ab45-38ba61d6c663 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:33.397261 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69778522-d2be-4e28-ab45-38ba61d6c663 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:33.397539 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69778522-d2be-4e28-ab45-38ba61d6c663 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:33.398837 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afdd847-5fe7-4b3d-ac96-a9fad7030e22 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:33.427817 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:33.444530 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75273c23-4117-4f2d-be43-dd8e14fe5661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:33.445185 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75273c23-4117-4f2d-be43-dd8e14fe5661 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:33.446122 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75273c23-4117-4f2d-be43-dd8e14fe5661 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:33.446122 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75273c23-4117-4f2d-be43-dd8e14fe5661 None admin] PATH_INFO: `/v3/groups/bf403365321e46f48e3808469483a22e` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:33.447098 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75273c23-4117-4f2d-be43-dd8e14fe5661 None admin] RBAC: Authorizing `identity:get_group(group_id=bf403365321e46f48e3808469483a22e)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:33.448618 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75273c23-4117-4f2d-be43-dd8e14fe5661 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:33.450608 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 223/450] 10.222.0.18 () {56 vars in 1180 bytes} [Sat Aug 6 21:50:33 2022] GET /identity/v3/groups/bf403365321e46f48e3808469483a22e => generated 229 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:50:33.456962 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75273c23-4117-4f2d-be43-dd8e14fe5661 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:33.482497 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69778522-d2be-4e28-ab45-38ba61d6c663 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:33.483965 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69778522-d2be-4e28-ab45-38ba61d6c663 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:33.553396 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:33.559610 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75273c23-4117-4f2d-be43-dd8e14fe5661 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:33.581316 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 228/451] 10.222.0.18 () {58 vars in 1282 bytes} [Sat Aug 6 21:50:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 06 21:50:33.586946 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69778522-d2be-4e28-ab45-38ba61d6c663 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:33.593926 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4c2a86-3547-4b52-83dc-b9d557100aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:33.594527 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c2a86-3547-4b52-83dc-b9d557100aa1 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:33.594799 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c2a86-3547-4b52-83dc-b9d557100aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:33.595677 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c2a86-3547-4b52-83dc-b9d557100aa1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:33.596434 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4c2a86-3547-4b52-83dc-b9d557100aa1 None admin] RBAC: Adding query filter params (group.id=bf403365321e46f48e3808469483a22e, role.id=4b8dcf0491fd4072acc69fd894bf77d0, scope.project.id=c285ce4925f04fb5acce87c789a71e83) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:33.596744 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4c2a86-3547-4b52-83dc-b9d557100aa1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:33.598170 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4c2a86-3547-4b52-83dc-b9d557100aa1 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:33.603750 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 224/452] 10.222.0.18 () {56 vars in 1376 bytes} [Sat Aug 6 21:50:33 2022] GET /identity/v3/role_assignments?group.id=bf403365321e46f48e3808469483a22e&scope.project.id=c285ce4925f04fb5acce87c789a71e83&role.id=4b8dcf0491fd4072acc69fd894bf77d0 => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:50:33.641923 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:33.692140 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69778522-d2be-4e28-ab45-38ba61d6c663 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:33.727091 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6d5f0f-4fbc-43e4-abf9-3a8be512d9a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:33.727725 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6d5f0f-4fbc-43e4-abf9-3a8be512d9a3 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:33.728004 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6d5f0f-4fbc-43e4-abf9-3a8be512d9a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:33.728278 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6d5f0f-4fbc-43e4-abf9-3a8be512d9a3 None admin] PATH_INFO: `/v3/projects/0bfd7af746994807b3d8de6f7292ef02` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:33.729454 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6d5f0f-4fbc-43e4-abf9-3a8be512d9a3 None admin] RBAC: Authorizing `identity:get_project(project_id=0bfd7af746994807b3d8de6f7292ef02)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:33.730913 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6d5f0f-4fbc-43e4-abf9-3a8be512d9a3 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:33.732860 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 229/453] 10.222.0.18 () {56 vars in 1186 bytes} [Sat Aug 6 21:50:33 2022] GET /identity/v3/projects/0bfd7af746994807b3d8de6f7292ef02 => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:50:33.739603 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4c2a86-3547-4b52-83dc-b9d557100aa1 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:33.837258 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4c2a86-3547-4b52-83dc-b9d557100aa1 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:33.869739 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35624415-b5b0-4dac-b2a4-88af46439fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:33.870354 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35624415-b5b0-4dac-b2a4-88af46439fc0 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:33.870628 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35624415-b5b0-4dac-b2a4-88af46439fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:33.870897 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35624415-b5b0-4dac-b2a4-88af46439fc0 None admin] PATH_INFO: `/v3/groups/c415347e96884e65a7ab87b53f4860d2` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:33.871975 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35624415-b5b0-4dac-b2a4-88af46439fc0 None admin] RBAC: Authorizing `identity:get_group(group_id=c415347e96884e65a7ab87b53f4860d2)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:33.873426 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35624415-b5b0-4dac-b2a4-88af46439fc0 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:33.874958 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 225/454] 10.222.0.18 () {56 vars in 1180 bytes} [Sat Aug 6 21:50:33 2022] GET /identity/v3/groups/c415347e96884e65a7ab87b53f4860d2 => generated 226 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:33.881073 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6d5f0f-4fbc-43e4-abf9-3a8be512d9a3 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:33.976751 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6d5f0f-4fbc-43e4-abf9-3a8be512d9a3 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:33.997727 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:34.011399 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25167d0b-a60c-46ad-8b93-6639b76291c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:34.013049 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25167d0b-a60c-46ad-8b93-6639b76291c2 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:34.013049 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25167d0b-a60c-46ad-8b93-6639b76291c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:34.013049 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25167d0b-a60c-46ad-8b93-6639b76291c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:34.013278 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25167d0b-a60c-46ad-8b93-6639b76291c2 None admin] RBAC: Adding query filter params (group.id=c415347e96884e65a7ab87b53f4860d2, role.id=93c058ac49e242fd8517c80c552dea2d, scope.project.id=0bfd7af746994807b3d8de6f7292ef02) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:34.013551 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25167d0b-a60c-46ad-8b93-6639b76291c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:34.014983 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25167d0b-a60c-46ad-8b93-6639b76291c2 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:34.020601 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 230/455] 10.222.0.18 () {56 vars in 1376 bytes} [Sat Aug 6 21:50:33 2022] GET /identity/v3/role_assignments?group.id=c415347e96884e65a7ab87b53f4860d2&scope.project.id=0bfd7af746994807b3d8de6f7292ef02&role.id=93c058ac49e242fd8517c80c552dea2d => generated 611 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:36.486876 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc6323d-f311-4a3b-96d7-112e5119035f None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:36.487407 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc6323d-f311-4a3b-96d7-112e5119035f None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:36.487407 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc6323d-f311-4a3b-96d7-112e5119035f None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:36.488791 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 226/456] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:50:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:50:36.494110 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fd6b77-3e99-4c8d-933b-ec7167a82176 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:36.494573 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fd6b77-3e99-4c8d-933b-ec7167a82176 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:36.494651 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fd6b77-3e99-4c8d-933b-ec7167a82176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:36.516443 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-b8fd6b77-3e99-4c8d-933b-ec7167a82176 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:50:36.518351 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-b8fd6b77-3e99-4c8d-933b-ec7167a82176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:50:36.608258 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 231/457] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:50:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 21:50:36.643964 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc6323d-f311-4a3b-96d7-112e5119035f None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:36.686208 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:36.790097 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc6323d-f311-4a3b-96d7-112e5119035f None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:36.831247 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a910bc01-8060-4275-9071-432f3cf86d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:36.831891 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a910bc01-8060-4275-9071-432f3cf86d0d None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:36.832177 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a910bc01-8060-4275-9071-432f3cf86d0d None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:36.832469 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a910bc01-8060-4275-9071-432f3cf86d0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:36.850402 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a910bc01-8060-4275-9071-432f3cf86d0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:36.851862 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a910bc01-8060-4275-9071-432f3cf86d0d None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:36.872084 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-a910bc01-8060-4275-9071-432f3cf86d0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:50:36.872084 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:36.872084 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:36.872084 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:36.872084 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:36.872084 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:36.872084 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:36.872084 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:36.872084 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:36.872084 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:36.872084 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:36.872084 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:36.872084 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:50:36.872084 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:50:36.872084 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:50:36.872084 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:50:36.873070 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:36.873070 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:36.873070 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:36.873070 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:36.873070 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:36.873070 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:36.873070 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:36.873070 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:36.873070 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:36.873070 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:36.873070 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:36.873070 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:36.873070 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:36.873070 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:36.873070 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:36.873070 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:36.873070 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:50:36.874517 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:50:36.874517 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:50:36.874517 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:50:36.874517 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 227/458] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:50:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:36.879681 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fd6b77-3e99-4c8d-933b-ec7167a82176 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:36.914303 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab363c0-e101-4f78-a44a-3e0aaaa1ce32 None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:36.914641 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab363c0-e101-4f78-a44a-3e0aaaa1ce32 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:36.914852 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab363c0-e101-4f78-a44a-3e0aaaa1ce32 None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:36.916449 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 228/459] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:50:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:50:36.929705 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4357803-edce-4a8d-a521-1fd22ad7f38e None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:36.930009 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4357803-edce-4a8d-a521-1fd22ad7f38e None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:36.930250 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4357803-edce-4a8d-a521-1fd22ad7f38e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:36.963167 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-b4357803-edce-4a8d-a521-1fd22ad7f38e None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:50:36.964376 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-b4357803-edce-4a8d-a521-1fd22ad7f38e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:50:37.039519 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fd6b77-3e99-4c8d-933b-ec7167a82176 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:37.060780 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:37.083301 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39e970f-fafe-45bf-920b-016b912d8cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:37.083917 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39e970f-fafe-45bf-920b-016b912d8cc9 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:37.084198 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39e970f-fafe-45bf-920b-016b912d8cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:37.084506 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39e970f-fafe-45bf-920b-016b912d8cc9 None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:37.087286 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39e970f-fafe-45bf-920b-016b912d8cc9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:37.087596 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39e970f-fafe-45bf-920b-016b912d8cc9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:37.089013 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39e970f-fafe-45bf-920b-016b912d8cc9 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:37.103542 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 229/460] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:50:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 21:50:37.113467 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 232/461] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:50:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:37.118564 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4357803-edce-4a8d-a521-1fd22ad7f38e None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:37.134902 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:37.153844 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39e970f-fafe-45bf-920b-016b912d8cc9 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:37.244840 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4357803-edce-4a8d-a521-1fd22ad7f38e None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:37.260823 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39e970f-fafe-45bf-920b-016b912d8cc9 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:37.279420 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a97a11-2532-4209-9027-d775059736bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:37.280031 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a97a11-2532-4209-9027-d775059736bd None admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:37.280310 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a97a11-2532-4209-9027-d775059736bd None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:37.280610 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a97a11-2532-4209-9027-d775059736bd None admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:37.281259 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a97a11-2532-4209-9027-d775059736bd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:37.283477 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a97a11-2532-4209-9027-d775059736bd None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:37.298233 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2d668d-4f01-4a94-8b95-67993b58959b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:37.298858 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2d668d-4f01-4a94-8b95-67993b58959b None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:37.299137 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2d668d-4f01-4a94-8b95-67993b58959b None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:37.299409 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2d668d-4f01-4a94-8b95-67993b58959b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:37.310111 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2d668d-4f01-4a94-8b95-67993b58959b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:37.311560 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2d668d-4f01-4a94-8b95-67993b58959b None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:37.327097 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-9b2d668d-4f01-4a94-8b95-67993b58959b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:50:37.327097 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:37.327097 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:37.327097 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:37.327097 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:37.327097 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:37.327097 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:37.327097 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:37.327097 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:37.327097 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:37.327097 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:37.327097 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:37.327097 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:50:37.327097 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:50:37.327097 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:50:37.327097 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:50:37.328157 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:37.328157 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:37.328157 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:37.328157 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:37.328157 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:37.328157 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:37.328157 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:37.328157 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:37.328157 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:37.328157 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:37.328157 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:37.328157 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:37.328157 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:37.328157 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:37.328157 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:37.328157 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:37.328157 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:50:37.329755 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:50:37.329755 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:50:37.329755 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:50:37.332545 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 233/462] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:50:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 21:50:37.334153 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9267a6a6-c9e8-4077-bc3c-ee40d70b3a26 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:37.334451 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9267a6a6-c9e8-4077-bc3c-ee40d70b3a26 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:37.334725 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9267a6a6-c9e8-4077-bc3c-ee40d70b3a26 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:37.336019 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 234/463] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:50:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:50:37.340962 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97555e0f-ba70-4642-a9e0-3f9060868822 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:37.341247 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97555e0f-ba70-4642-a9e0-3f9060868822 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:37.341520 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97555e0f-ba70-4642-a9e0-3f9060868822 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:37.342690 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 235/464] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:50:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:50:37.343373 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97555e0f-ba70-4642-a9e0-3f9060868822 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:37.377955 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:37.392868 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 230/465] 10.222.0.18 () {60 vars in 1132 bytes} [Sat Aug 6 21:50:37 2022] POST /identity/v3/users => generated 246 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 21:50:37.394240 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d375ffe6-fee1-4dd3-80db-9ea738f2e418 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:37.394533 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d375ffe6-fee1-4dd3-80db-9ea738f2e418 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:37.394880 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d375ffe6-fee1-4dd3-80db-9ea738f2e418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:37.415119 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-d375ffe6-fee1-4dd3-80db-9ea738f2e418 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:50:37.416372 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-d375ffe6-fee1-4dd3-80db-9ea738f2e418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:50:37.476134 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97555e0f-ba70-4642-a9e0-3f9060868822 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:37.491051 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:37.505201 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 231/466] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:50:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 21:50:37.507007 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5abcb40-8c36-4491-a184-d0df85daf002 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:37.507541 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5abcb40-8c36-4491-a184-d0df85daf002 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:37.508069 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5abcb40-8c36-4491-a184-d0df85daf002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:37.531840 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-c5abcb40-8c36-4491-a184-d0df85daf002 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:50:37.533873 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-c5abcb40-8c36-4491-a184-d0df85daf002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:50:37.543598 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ba0c2e-5c16-4d64-87e7-c55fc3bfc18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:37.544346 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ba0c2e-5c16-4d64-87e7-c55fc3bfc18b None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:37.544659 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ba0c2e-5c16-4d64-87e7-c55fc3bfc18b None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:37.544940 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ba0c2e-5c16-4d64-87e7-c55fc3bfc18b None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:37.545604 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ba0c2e-5c16-4d64-87e7-c55fc3bfc18b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:37.545913 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ba0c2e-5c16-4d64-87e7-c55fc3bfc18b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:37.547354 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ba0c2e-5c16-4d64-87e7-c55fc3bfc18b None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:37.567270 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 236/467] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:50:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:37.568264 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ba0c2e-5c16-4d64-87e7-c55fc3bfc18b None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:37.629446 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 232/468] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:50:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 21:50:37.630371 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5abcb40-8c36-4491-a184-d0df85daf002 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:37.676626 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:37.697651 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ba0c2e-5c16-4d64-87e7-c55fc3bfc18b None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:37.734907 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f4059c-5698-4ceb-b11a-d7e068fc7a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:37.736093 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f4059c-5698-4ceb-b11a-d7e068fc7a24 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:37.736372 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f4059c-5698-4ceb-b11a-d7e068fc7a24 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:37.736719 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f4059c-5698-4ceb-b11a-d7e068fc7a24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:37.750280 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5abcb40-8c36-4491-a184-d0df85daf002 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:37.750467 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f4059c-5698-4ceb-b11a-d7e068fc7a24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:37.753517 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f4059c-5698-4ceb-b11a-d7e068fc7a24 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:37.772805 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-d7f4059c-5698-4ceb-b11a-d7e068fc7a24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:50:37.772805 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:37.772805 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:37.772805 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:37.772805 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:37.772805 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:37.772805 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:37.772805 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:37.772805 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:37.772805 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:37.772805 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:37.772805 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:37.772805 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:50:37.772805 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:50:37.772805 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:50:37.772805 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:50:37.773753 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:37.773753 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:37.773753 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:37.773753 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:37.773753 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:37.773753 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:37.773753 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:37.773753 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:37.773753 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:37.773753 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:37.773753 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:37.773753 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:37.773753 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:37.773753 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:37.773753 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:37.773753 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:37.773753 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:50:37.774598 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:50:37.774598 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:50:37.774598 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:50:37.776615 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 237/469] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:50:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:37.777630 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f4059c-5698-4ceb-b11a-d7e068fc7a24 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:37.782670 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:37.794010 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8026f0ed-f3f5-414f-a01a-4be9bcbc0e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:37.795093 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8026f0ed-f3f5-414f-a01a-4be9bcbc0e23 None admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:37.795352 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8026f0ed-f3f5-414f-a01a-4be9bcbc0e23 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:37.795931 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8026f0ed-f3f5-414f-a01a-4be9bcbc0e23 None admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:37.796707 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8026f0ed-f3f5-414f-a01a-4be9bcbc0e23 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:37.798904 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8026f0ed-f3f5-414f-a01a-4be9bcbc0e23 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:37.877340 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 233/470] 10.222.0.18 () {60 vars in 1132 bytes} [Sat Aug 6 21:50:37 2022] POST /identity/v3/users => generated 248 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 21:50:37.884065 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8026f0ed-f3f5-414f-a01a-4be9bcbc0e23 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:37.890449 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f4059c-5698-4ceb-b11a-d7e068fc7a24 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:37.934368 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd7d186-26ed-4d9b-ab78-1e7b61f70e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:37.935293 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd7d186-26ed-4d9b-ab78-1e7b61f70e02 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:37.936186 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd7d186-26ed-4d9b-ab78-1e7b61f70e02 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:37.936643 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd7d186-26ed-4d9b-ab78-1e7b61f70e02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:37.946448 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:37.951591 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd7d186-26ed-4d9b-ab78-1e7b61f70e02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:37.953848 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd7d186-26ed-4d9b-ab78-1e7b61f70e02 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:37.977473 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-8cd7d186-26ed-4d9b-ab78-1e7b61f70e02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:50:37.977473 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:37.977473 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:37.977473 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:37.977473 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:37.977473 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:37.977473 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:37.977473 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:37.977473 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:37.977473 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:37.977473 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:37.977473 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:37.977473 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:50:37.977473 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:50:37.977473 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:50:37.977473 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:50:37.978472 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:37.978472 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:37.978472 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:37.978472 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:37.978472 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:37.978472 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:37.978472 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:37.978472 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:37.978472 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:37.978472 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:37.978472 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:37.978472 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:37.978472 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:37.978472 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:37.978472 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:37.978472 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:37.978472 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:50:37.979329 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:50:37.979329 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:50:37.979329 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:50:37.980220 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 238/471] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:50:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:37.987539 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd7d186-26ed-4d9b-ab78-1e7b61f70e02 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:38.093214 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8026f0ed-f3f5-414f-a01a-4be9bcbc0e23 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:38.142428 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d957c52-705e-4e79-95be-277d38b5a58d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:38.143331 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d957c52-705e-4e79-95be-277d38b5a58d None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:38.143669 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d957c52-705e-4e79-95be-277d38b5a58d None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:38.143996 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d957c52-705e-4e79-95be-277d38b5a58d None admin] PATH_INFO: `/v3/domains` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:38.145119 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d957c52-705e-4e79-95be-277d38b5a58d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:38.145437 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d957c52-705e-4e79-95be-277d38b5a58d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:38.147434 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d957c52-705e-4e79-95be-277d38b5a58d None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:38.155686 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd7d186-26ed-4d9b-ab78-1e7b61f70e02 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:38.161865 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 234/472] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:50:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:38.163111 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc9213-4b0d-45df-82e6-b1055a4b4976 None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:38.163379 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc9213-4b0d-45df-82e6-b1055a4b4976 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:38.163664 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc9213-4b0d-45df-82e6-b1055a4b4976 None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:38.165376 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 235/473] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:50:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:50:38.170094 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecc9213-4b0d-45df-82e6-b1055a4b4976 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:38.207431 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019a76ef-8036-4851-ad44-47a71c57e00e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:38.208040 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019a76ef-8036-4851-ad44-47a71c57e00e None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:38.208318 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019a76ef-8036-4851-ad44-47a71c57e00e None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:38.208878 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019a76ef-8036-4851-ad44-47a71c57e00e None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:38.209286 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019a76ef-8036-4851-ad44-47a71c57e00e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:38.209587 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019a76ef-8036-4851-ad44-47a71c57e00e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:38.211111 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019a76ef-8036-4851-ad44-47a71c57e00e None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:38.223568 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:38.229868 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 239/474] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:50:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:38.230987 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42679613-b926-42f0-afa6-05eb36936d8a None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:38.231274 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42679613-b926-42f0-afa6-05eb36936d8a None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:38.231545 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42679613-b926-42f0-afa6-05eb36936d8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:38.252038 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-42679613-b926-42f0-afa6-05eb36936d8a None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:50:38.253327 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-42679613-b926-42f0-afa6-05eb36936d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:50:38.306079 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecc9213-4b0d-45df-82e6-b1055a4b4976 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:38.316755 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:38.343893 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 240/475] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:50:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 21:50:38.345808 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42679613-b926-42f0-afa6-05eb36936d8a None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:38.355691 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64112611-c269-4f61-ac7e-9fcc322475ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:38.362704 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64112611-c269-4f61-ac7e-9fcc322475ec None admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:38.363298 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64112611-c269-4f61-ac7e-9fcc322475ec None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:38.364101 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64112611-c269-4f61-ac7e-9fcc322475ec None admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:38.365191 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64112611-c269-4f61-ac7e-9fcc322475ec None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:38.366846 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64112611-c269-4f61-ac7e-9fcc322475ec None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:38.445196 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 236/476] 10.222.0.18 () {60 vars in 1132 bytes} [Sat Aug 6 21:50:38 2022] POST /identity/v3/users => generated 248 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 21:50:38.451156 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64112611-c269-4f61-ac7e-9fcc322475ec None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:38.507132 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42679613-b926-42f0-afa6-05eb36936d8a None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:38.581631 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221a969b-b89f-46e5-b15c-4d10ea68b139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:38.582390 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221a969b-b89f-46e5-b15c-4d10ea68b139 None admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:38.582574 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221a969b-b89f-46e5-b15c-4d10ea68b139 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:38.582849 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221a969b-b89f-46e5-b15c-4d10ea68b139 None admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:38.584723 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221a969b-b89f-46e5-b15c-4d10ea68b139 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:38.585195 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221a969b-b89f-46e5-b15c-4d10ea68b139 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:38.625037 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:38.631475 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64112611-c269-4f61-ac7e-9fcc322475ec None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:38.663935 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 241/477] 10.222.0.18 () {60 vars in 1132 bytes} [Sat Aug 6 21:50:38 2022] POST /identity/v3/users => generated 249 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 21:50:38.672658 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61768a2-9125-4e1d-ab85-9cfe3645f225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:38.674050 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61768a2-9125-4e1d-ab85-9cfe3645f225 None admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:38.674450 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61768a2-9125-4e1d-ab85-9cfe3645f225 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:38.674738 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61768a2-9125-4e1d-ab85-9cfe3645f225 None admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:38.675430 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61768a2-9125-4e1d-ab85-9cfe3645f225 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:38.676983 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61768a2-9125-4e1d-ab85-9cfe3645f225 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:38.734301 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.sql.core [None req-f61768a2-9125-4e1d-ab85-9cfe3645f225 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 06 21:50:38.734301 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 06 21:50:38.734301 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [parameters: {'id': 'b6ec28d982dd45f481a3686547f6a1b3', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 06 21:50:38.734301 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80423) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 06 21:50:38.754693 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-f61768a2-9125-4e1d-ab85-9cfe3645f225 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 06 21:50:38.754693 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:38.754693 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 06 21:50:38.754693 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 06 21:50:38.754693 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 06 21:50:38.754693 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 06 21:50:38.754693 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 06 21:50:38.754693 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application result = self._query(query) Aug 06 21:50:38.754693 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 06 21:50:38.754693 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application conn.query(q) Aug 06 21:50:38.754693 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 06 21:50:38.754693 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 06 21:50:38.754693 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 06 21:50:38.754693 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application result.read() Aug 06 21:50:38.754693 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 06 21:50:38.755662 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 06 21:50:38.755662 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 06 21:50:38.755662 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 06 21:50:38.755662 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 06 21:50:38.755662 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 06 21:50:38.755662 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 06 21:50:38.755662 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 06 21:50:38.755662 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 06 21:50:38.755662 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:50:38.755662 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 06 21:50:38.755662 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:50:38.755662 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:38.755662 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 06 21:50:38.755662 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 06 21:50:38.755662 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 06 21:50:38.755662 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return ref.to_dict() Aug 06 21:50:38.755662 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 06 21:50:38.755662 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application next(self.gen) Aug 06 21:50:38.756686 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 06 21:50:38.756686 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application yield resource Aug 06 21:50:38.756686 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 06 21:50:38.756686 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application next(self.gen) Aug 06 21:50:38.756686 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 06 21:50:38.756686 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application self.session.rollback() Aug 06 21:50:38.756686 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 06 21:50:38.756686 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application self.force_reraise() Aug 06 21:50:38.756686 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 06 21:50:38.756686 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise self.value Aug 06 21:50:38.756686 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 06 21:50:38.756686 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 06 21:50:38.756686 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 06 21:50:38.756686 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application session.commit() Aug 06 21:50:38.756686 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 06 21:50:38.756686 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 06 21:50:38.756686 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 06 21:50:38.757641 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application self._prepare_impl() Aug 06 21:50:38.757641 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 06 21:50:38.757641 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application self.session.flush() Aug 06 21:50:38.757641 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 06 21:50:38.757641 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application self._flush(objects) Aug 06 21:50:38.757641 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 06 21:50:38.757641 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 06 21:50:38.757641 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 06 21:50:38.757641 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application compat.raise_( Aug 06 21:50:38.757641 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 06 21:50:38.757641 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception Aug 06 21:50:38.757641 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 06 21:50:38.757641 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application flush_context.execute() Aug 06 21:50:38.757641 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 06 21:50:38.757641 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rec.execute(self) Aug 06 21:50:38.757641 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 06 21:50:38.757641 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 06 21:50:38.758550 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 06 21:50:38.758550 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 06 21:50:38.758550 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 06 21:50:38.758550 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 06 21:50:38.758550 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 06 21:50:38.758550 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 06 21:50:38.758550 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 06 21:50:38.758550 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 06 21:50:38.758550 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 06 21:50:38.758550 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 06 21:50:38.758550 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 06 21:50:38.758550 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 06 21:50:38.758550 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 06 21:50:38.758550 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 06 21:50:38.758550 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 06 21:50:38.758550 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception Aug 06 21:50:38.759356 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 06 21:50:38.759356 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 06 21:50:38.759356 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 06 21:50:38.759356 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 06 21:50:38.759356 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 06 21:50:38.759356 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application result = self._query(query) Aug 06 21:50:38.759356 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 06 21:50:38.759356 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application conn.query(q) Aug 06 21:50:38.759356 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 06 21:50:38.759356 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 06 21:50:38.759356 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 06 21:50:38.759356 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application result.read() Aug 06 21:50:38.759356 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 06 21:50:38.759356 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 06 21:50:38.759356 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 06 21:50:38.759356 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 06 21:50:38.760140 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 06 21:50:38.760140 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 06 21:50:38.760140 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 06 21:50:38.760140 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 06 21:50:38.760140 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 06 21:50:38.760140 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 06 21:50:38.760140 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [parameters: {'id': 'b6ec28d982dd45f481a3686547f6a1b3', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 06 21:50:38.760140 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 06 21:50:38.760140 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:50:38.760140 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 21:50:38.760140 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:50:38.760140 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:38.760140 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:38.760140 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:38.760140 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:38.760140 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:38.762787 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:38.762787 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:38.762787 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:38.762787 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:38.762787 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:38.762787 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:38.762787 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 06 21:50:38.762787 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 06 21:50:38.762787 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:38.762787 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:38.762787 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 06 21:50:38.762787 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 06 21:50:38.762787 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 06 21:50:38.762787 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 06 21:50:38.762787 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 06 21:50:38.762787 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:50:38.764859 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 237/478] 10.222.0.18 () {60 vars in 1131 bytes} [Sat Aug 6 21:50:38 2022] POST /identity/v3/roles => generated 144 bytes in 311 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:38.768434 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221a969b-b89f-46e5-b15c-4d10ea68b139 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:38.820679 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:38.906219 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221a969b-b89f-46e5-b15c-4d10ea68b139 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:38.942028 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c6e96e-0dc1-42e5-a870-7a05c27368c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:38.942665 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c6e96e-0dc1-42e5-a870-7a05c27368c5 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:38.942941 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c6e96e-0dc1-42e5-a870-7a05c27368c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:38.943211 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c6e96e-0dc1-42e5-a870-7a05c27368c5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:38.953833 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c6e96e-0dc1-42e5-a870-7a05c27368c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:50:38.960420 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-01c6e96e-0dc1-42e5-a870-7a05c27368c5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 21:50:38.960420 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:38.960420 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:38.960420 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:38.960420 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:38.960420 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:38.960420 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:38.960420 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:38.960420 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:38.960420 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:38.960420 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:38.960420 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:38.960420 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:50:38.960420 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:50:38.960420 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:50:38.960420 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:50:38.962462 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:50:38.962462 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:50:38.962462 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:50:38.962462 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:50:38.962462 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:38.962462 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:38.962462 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:38.962462 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:38.962462 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:38.962462 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:38.962462 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:38.962462 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:38.962462 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:38.962462 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:38.962462 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:38.962462 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:38.964364 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:38.964364 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:38.964364 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:38.964364 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:38.964364 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:50:38.964364 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:50:38.964364 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:50:38.964364 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:50:38.964364 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:50:38.964364 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:50:38.964364 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 21:50:38.964364 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:50:38.964364 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 242/479] 10.222.0.18 () {56 vars in 1114 bytes} [Sat Aug 6 21:50:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:38.969558 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61768a2-9125-4e1d-ab85-9cfe3645f225 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:39.073579 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61768a2-9125-4e1d-ab85-9cfe3645f225 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:39.108092 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab9d281-ee37-448a-83bc-ca34180209d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:39.108743 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab9d281-ee37-448a-83bc-ca34180209d6 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:39.111159 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab9d281-ee37-448a-83bc-ca34180209d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:39.111159 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab9d281-ee37-448a-83bc-ca34180209d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:39.111159 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab9d281-ee37-448a-83bc-ca34180209d6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:39.111159 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab9d281-ee37-448a-83bc-ca34180209d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:39.111749 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab9d281-ee37-448a-83bc-ca34180209d6 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:39.121075 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 238/480] 10.222.0.18 () {56 vars in 1111 bytes} [Sat Aug 6 21:50:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:39.854046 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286ab6f7-a560-44a5-a3d1-8c8e7d61e40d None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:39.854591 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286ab6f7-a560-44a5-a3d1-8c8e7d61e40d None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:39.854591 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286ab6f7-a560-44a5-a3d1-8c8e7d61e40d None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:39.855961 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 243/481] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:50:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:50:39.873926 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a161284-f11c-470f-95ec-327caffaed83 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:39.874425 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a161284-f11c-470f-95ec-327caffaed83 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:39.874496 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a161284-f11c-470f-95ec-327caffaed83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:39.901412 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-9a161284-f11c-470f-95ec-327caffaed83 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:50:39.902605 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-9a161284-f11c-470f-95ec-327caffaed83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:50:39.921085 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:39.975589 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 239/482] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:50:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 21:50:40.009736 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286ab6f7-a560-44a5-a3d1-8c8e7d61e40d None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:40.138776 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:40.149078 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286ab6f7-a560-44a5-a3d1-8c8e7d61e40d None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:40.190748 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b70a6d-6323-4a0c-91c0-51ac1572ccf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:40.191378 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b70a6d-6323-4a0c-91c0-51ac1572ccf5 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:40.191657 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b70a6d-6323-4a0c-91c0-51ac1572ccf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:40.191932 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b70a6d-6323-4a0c-91c0-51ac1572ccf5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:40.203374 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b70a6d-6323-4a0c-91c0-51ac1572ccf5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:50:40.208719 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-92b70a6d-6323-4a0c-91c0-51ac1572ccf5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 21:50:40.208719 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:40.208719 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:40.208719 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:40.208719 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:40.208719 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:40.208719 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:40.208719 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:40.208719 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:40.208719 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:40.208719 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:40.208719 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:40.208719 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:50:40.208719 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:50:40.208719 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:50:40.208719 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:50:40.209685 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:50:40.209685 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:50:40.209685 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:50:40.209685 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:50:40.209685 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:40.209685 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:40.209685 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:40.209685 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:40.209685 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:40.209685 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:40.209685 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:40.209685 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:40.209685 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:40.209685 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:40.209685 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:40.209685 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:40.210495 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:40.210495 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:40.210495 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:40.210495 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:40.210495 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:50:40.210495 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:50:40.210495 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:50:40.210495 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:50:40.210495 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:50:40.210495 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:50:40.210495 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 21:50:40.210495 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:50:40.210495 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 244/483] 10.222.0.18 () {56 vars in 1102 bytes} [Sat Aug 6 21:50:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:40.215747 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a161284-f11c-470f-95ec-327caffaed83 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:40.350427 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a161284-f11c-470f-95ec-327caffaed83 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:40.386517 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df0153a-93f3-4815-85cb-9e0d824e1925 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:40.386794 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df0153a-93f3-4815-85cb-9e0d824e1925 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:40.387064 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df0153a-93f3-4815-85cb-9e0d824e1925 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:40.389365 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 245/484] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:50:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 21:50:40.397023 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dfdcf5-bcba-4b86-b834-6505cf75ee4f None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:40.397305 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dfdcf5-bcba-4b86-b834-6505cf75ee4f None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:40.397576 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dfdcf5-bcba-4b86-b834-6505cf75ee4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:40.404901 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810d5cf5-98e8-48ae-b872-a5df0af53e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:40.405520 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d5cf5-98e8-48ae-b872-a5df0af53e96 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:40.405798 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d5cf5-98e8-48ae-b872-a5df0af53e96 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:40.406072 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d5cf5-98e8-48ae-b872-a5df0af53e96 None admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:40.406754 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810d5cf5-98e8-48ae-b872-a5df0af53e96 None admin] RBAC: Adding query filter params (name=service) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:40.407577 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810d5cf5-98e8-48ae-b872-a5df0af53e96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:40.408991 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810d5cf5-98e8-48ae-b872-a5df0af53e96 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:40.427831 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-06dfdcf5-bcba-4b86-b834-6505cf75ee4f None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:50:40.428921 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 240/485] 10.222.0.18 () {56 vars in 1103 bytes} [Sat Aug 6 21:50:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:40.429430 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-06dfdcf5-bcba-4b86-b834-6505cf75ee4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:50:40.432095 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810d5cf5-98e8-48ae-b872-a5df0af53e96 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:40.464742 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:40.527938 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 246/486] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:50:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 21:50:40.573430 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dfdcf5-bcba-4b86-b834-6505cf75ee4f None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:40.584365 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810d5cf5-98e8-48ae-b872-a5df0af53e96 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:40.622639 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6e7610-ca41-47f2-be56-d360116caae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:40.623615 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6e7610-ca41-47f2-be56-d360116caae9 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:40.624096 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6e7610-ca41-47f2-be56-d360116caae9 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:40.624617 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6e7610-ca41-47f2-be56-d360116caae9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:40.635822 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6e7610-ca41-47f2-be56-d360116caae9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:40.637545 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6e7610-ca41-47f2-be56-d360116caae9 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:40.654658 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-be6e7610-ca41-47f2-be56-d360116caae9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:50:40.654658 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:40.654658 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:40.654658 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:40.654658 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:40.654658 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:40.654658 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:40.654658 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:40.654658 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:40.654658 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:40.654658 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:40.654658 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:40.654658 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:50:40.654658 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:50:40.654658 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:50:40.654658 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:50:40.655582 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:40.655582 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:40.655582 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:40.655582 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:40.655582 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:40.655582 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:40.655582 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:40.655582 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:40.655582 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:40.655582 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:40.655582 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:40.655582 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:40.655582 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:40.655582 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:40.655582 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:40.655582 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:40.655582 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:50:40.656424 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:50:40.656424 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:50:40.656424 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:50:40.659492 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 241/487] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:50:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:40.665878 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6e7610-ca41-47f2-be56-d360116caae9 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:40.731322 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:40.749240 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dfdcf5-bcba-4b86-b834-6505cf75ee4f None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:40.791275 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07646482-5e4c-469b-b3c3-e3ba41efd4b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:40.792645 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07646482-5e4c-469b-b3c3-e3ba41efd4b5 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:40.793135 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07646482-5e4c-469b-b3c3-e3ba41efd4b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:40.793610 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07646482-5e4c-469b-b3c3-e3ba41efd4b5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:40.807940 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:40.810516 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07646482-5e4c-469b-b3c3-e3ba41efd4b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:50:40.817033 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-07646482-5e4c-469b-b3c3-e3ba41efd4b5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 21:50:40.817033 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:40.817033 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:40.817033 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:40.817033 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:40.817033 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:40.817033 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:40.817033 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:40.817033 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:40.817033 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:40.817033 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:40.817033 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:40.817033 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:50:40.817033 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:50:40.817033 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:50:40.817033 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:50:40.820094 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:50:40.820094 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:50:40.820094 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:50:40.820094 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:50:40.820094 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:40.820094 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:40.820094 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:40.820094 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:40.820094 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:40.820094 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:40.820094 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:40.820094 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:40.820094 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:40.820094 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:40.820094 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:40.820094 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:40.821034 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:40.821034 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:40.821034 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:40.821034 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:40.821034 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:50:40.821034 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:50:40.821034 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:50:40.821034 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:50:40.821034 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:50:40.821034 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:50:40.821034 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 21:50:40.821034 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:50:40.821034 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 247/488] 10.222.0.18 () {56 vars in 1102 bytes} [Sat Aug 6 21:50:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:40.838403 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07646482-5e4c-469b-b3c3-e3ba41efd4b5 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:40.879522 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6e7610-ca41-47f2-be56-d360116caae9 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:40.923899 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79eacd9-5b93-4e9a-8bce-ea895600a6e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:40.924518 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79eacd9-5b93-4e9a-8bce-ea895600a6e0 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:40.924802 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79eacd9-5b93-4e9a-8bce-ea895600a6e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:40.925080 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79eacd9-5b93-4e9a-8bce-ea895600a6e0 None admin] PATH_INFO: `/v3/domains` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:40.925744 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79eacd9-5b93-4e9a-8bce-ea895600a6e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:40.926046 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79eacd9-5b93-4e9a-8bce-ea895600a6e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:40.927545 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79eacd9-5b93-4e9a-8bce-ea895600a6e0 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:40.949706 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 242/489] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:50:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:40.951678 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f0aa25-6b0b-49b7-bdc5-c3c7365695ef None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:40.951960 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f0aa25-6b0b-49b7-bdc5-c3c7365695ef None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:40.952231 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f0aa25-6b0b-49b7-bdc5-c3c7365695ef None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:40.958635 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 243/490] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:50:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:50:40.959446 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f0aa25-6b0b-49b7-bdc5-c3c7365695ef None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:40.989724 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07646482-5e4c-469b-b3c3-e3ba41efd4b5 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:41.026192 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1f0234-9dc8-4348-a3f4-59a2ff4e9f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:41.027253 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f0234-9dc8-4348-a3f4-59a2ff4e9f44 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:41.027600 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f0234-9dc8-4348-a3f4-59a2ff4e9f44 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:41.027939 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f0234-9dc8-4348-a3f4-59a2ff4e9f44 None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:41.028715 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1f0234-9dc8-4348-a3f4-59a2ff4e9f44 None admin] RBAC: Adding query filter params (name=service) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:41.029084 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1f0234-9dc8-4348-a3f4-59a2ff4e9f44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:41.030553 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1f0234-9dc8-4348-a3f4-59a2ff4e9f44 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:41.039608 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 248/491] 10.222.0.18 () {56 vars in 1103 bytes} [Sat Aug 6 21:50:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:50:41.040968 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84690d3b-6c22-4c20-bd3d-129398118b6a None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:41.041324 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84690d3b-6c22-4c20-bd3d-129398118b6a None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:41.041668 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84690d3b-6c22-4c20-bd3d-129398118b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:41.064434 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-84690d3b-6c22-4c20-bd3d-129398118b6a None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:50:41.064434 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-84690d3b-6c22-4c20-bd3d-129398118b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:50:41.077806 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:41.079455 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f0aa25-6b0b-49b7-bdc5-c3c7365695ef None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:41.113236 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:41.122852 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee8bede-0fd8-4e07-8aa8-ba67ccccc518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:41.123494 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee8bede-0fd8-4e07-8aa8-ba67ccccc518 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:41.123783 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee8bede-0fd8-4e07-8aa8-ba67ccccc518 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:41.124058 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee8bede-0fd8-4e07-8aa8-ba67ccccc518 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:41.134316 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee8bede-0fd8-4e07-8aa8-ba67ccccc518 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:41.135774 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee8bede-0fd8-4e07-8aa8-ba67ccccc518 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:41.152504 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-bee8bede-0fd8-4e07-8aa8-ba67ccccc518 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 06 21:50:41.152504 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:41.152504 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:41.152504 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:41.152504 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:41.152504 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:41.152504 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:41.152504 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:41.152504 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:41.152504 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:41.152504 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:41.152504 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:41.152504 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 21:50:41.152504 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 21:50:41.152504 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 21:50:41.152504 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 21:50:41.153403 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:41.153403 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:41.153403 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 21:50:41.153403 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 21:50:41.153403 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 21:50:41.153403 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 21:50:41.153403 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:41.153403 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:41.153403 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:41.153403 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:41.153403 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:41.153403 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:41.153403 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:41.153403 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:41.153403 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:41.153403 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:41.153403 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:41.153403 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:41.154299 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:41.154299 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:41.154299 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 21:50:41.154299 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 21:50:41.154299 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 21:50:41.154299 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 21:50:41.154299 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 21:50:41.154299 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 21:50:41.154299 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 06 21:50:41.154299 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:50:41.154299 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 244/492] 10.222.0.18 () {56 vars in 1093 bytes} [Sat Aug 6 21:50:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:41.154948 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee8bede-0fd8-4e07-8aa8-ba67ccccc518 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:41.160270 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 249/493] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:50:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 21:50:41.161426 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84690d3b-6c22-4c20-bd3d-129398118b6a None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:41.286689 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee8bede-0fd8-4e07-8aa8-ba67ccccc518 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:41.309377 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84690d3b-6c22-4c20-bd3d-129398118b6a None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:41.330238 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dd01b9-e50e-45c1-a0f2-ee59e2057521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:41.331253 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dd01b9-e50e-45c1-a0f2-ee59e2057521 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:41.331789 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dd01b9-e50e-45c1-a0f2-ee59e2057521 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:41.332267 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dd01b9-e50e-45c1-a0f2-ee59e2057521 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:41.349145 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cff2d21-59b6-40db-ba09-fc9cdae076a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:41.349145 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cff2d21-59b6-40db-ba09-fc9cdae076a4 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:41.349145 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cff2d21-59b6-40db-ba09-fc9cdae076a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:41.349145 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cff2d21-59b6-40db-ba09-fc9cdae076a4 None admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:41.349531 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cff2d21-59b6-40db-ba09-fc9cdae076a4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:41.349531 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cff2d21-59b6-40db-ba09-fc9cdae076a4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:41.350721 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cff2d21-59b6-40db-ba09-fc9cdae076a4 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:41.355515 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dd01b9-e50e-45c1-a0f2-ee59e2057521 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:41.362732 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dd01b9-e50e-45c1-a0f2-ee59e2057521 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:41.370503 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:41.381683 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-f2dd01b9-e50e-45c1-a0f2-ee59e2057521 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:50:41.381683 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:41.381683 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:41.381683 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:41.381683 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:41.381683 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:41.381683 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:41.381683 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:41.381683 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:41.381683 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:41.381683 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:41.381683 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:41.381683 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:50:41.381683 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:50:41.381683 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:50:41.381683 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:50:41.382589 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:41.382589 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:41.382589 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:41.382589 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:41.382589 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:41.382589 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:41.382589 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:41.382589 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:41.382589 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:41.382589 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:41.382589 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:41.382589 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:41.382589 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:41.382589 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:41.382589 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:41.382589 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:41.382589 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:50:41.383430 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:50:41.383430 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:50:41.383430 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:50:41.387622 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 245/494] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:50:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:41.389963 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dd01b9-e50e-45c1-a0f2-ee59e2057521 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:41.401776 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 250/495] 10.222.0.18 () {56 vars in 1133 bytes} [Sat Aug 6 21:50:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:41.403574 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cff2d21-59b6-40db-ba09-fc9cdae076a4 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:41.435097 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:41.503815 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dd01b9-e50e-45c1-a0f2-ee59e2057521 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:41.542258 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b89e0ba-570d-48d4-b2e9-f4732e1791e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:41.543154 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b89e0ba-570d-48d4-b2e9-f4732e1791e2 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:41.543348 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b89e0ba-570d-48d4-b2e9-f4732e1791e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:41.543740 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b89e0ba-570d-48d4-b2e9-f4732e1791e2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:41.549951 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cff2d21-59b6-40db-ba09-fc9cdae076a4 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:41.557569 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b89e0ba-570d-48d4-b2e9-f4732e1791e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:50:41.563017 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-2b89e0ba-570d-48d4-b2e9-f4732e1791e2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 21:50:41.563017 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:41.563017 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:41.563017 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:41.563017 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:41.563017 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:41.563017 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:41.563017 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:41.563017 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:41.563017 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:41.563017 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:41.563017 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:41.563017 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:50:41.563017 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:50:41.563017 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:50:41.563017 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:50:41.564587 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:50:41.564587 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:50:41.564587 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:50:41.564587 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:50:41.564587 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:41.564587 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:41.564587 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:41.564587 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:41.564587 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:41.564587 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:41.564587 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:41.564587 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:41.564587 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:41.564587 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:41.564587 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:41.564587 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:41.565403 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:41.565403 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:41.565403 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:41.565403 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:41.565403 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:50:41.565403 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:50:41.565403 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:50:41.565403 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:50:41.565403 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:50:41.565403 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:50:41.565403 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 21:50:41.565403 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:50:41.569379 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 246/496] 10.222.0.18 () {56 vars in 1102 bytes} [Sat Aug 6 21:50:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:41.570097 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b89e0ba-570d-48d4-b2e9-f4732e1791e2 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:41.589398 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f5cf21-7e96-43df-a4e3-cb0a5dbea951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:41.590012 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f5cf21-7e96-43df-a4e3-cb0a5dbea951 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:41.590291 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f5cf21-7e96-43df-a4e3-cb0a5dbea951 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:41.590563 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f5cf21-7e96-43df-a4e3-cb0a5dbea951 None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:41.591204 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f5cf21-7e96-43df-a4e3-cb0a5dbea951 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:41.591501 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f5cf21-7e96-43df-a4e3-cb0a5dbea951 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:41.592932 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f5cf21-7e96-43df-a4e3-cb0a5dbea951 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:41.609486 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 251/497] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:50:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:41.609486 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4684ad-5b45-4e56-9bdc-19080acf9af8 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:41.609486 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4684ad-5b45-4e56-9bdc-19080acf9af8 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:41.609486 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4684ad-5b45-4e56-9bdc-19080acf9af8 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:41.609486 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 252/498] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:50:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:50:41.609486 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4684ad-5b45-4e56-9bdc-19080acf9af8 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:41.678518 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:41.679075 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:41.682841 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b89e0ba-570d-48d4-b2e9-f4732e1791e2 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:41.717969 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8973e05-5a68-4b32-937b-6891784c57bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:41.718667 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8973e05-5a68-4b32-937b-6891784c57bd None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:41.718951 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8973e05-5a68-4b32-937b-6891784c57bd None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:41.719226 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8973e05-5a68-4b32-937b-6891784c57bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:41.726284 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4684ad-5b45-4e56-9bdc-19080acf9af8 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:41.762770 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b05dfd1-95a7-4fc9-b55b-d7c71b2372c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:41.763379 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b05dfd1-95a7-4fc9-b55b-d7c71b2372c8 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:41.763657 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b05dfd1-95a7-4fc9-b55b-d7c71b2372c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:41.763928 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b05dfd1-95a7-4fc9-b55b-d7c71b2372c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:41.764617 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b05dfd1-95a7-4fc9-b55b-d7c71b2372c8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:41.764926 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b05dfd1-95a7-4fc9-b55b-d7c71b2372c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:41.766358 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b05dfd1-95a7-4fc9-b55b-d7c71b2372c8 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:41.775058 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 253/499] 10.222.0.18 () {56 vars in 1103 bytes} [Sat Aug 6 21:50:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:41.775981 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b05dfd1-95a7-4fc9-b55b-d7c71b2372c8 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:41.792661 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8973e05-5a68-4b32-937b-6891784c57bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:41.794328 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8973e05-5a68-4b32-937b-6891784c57bd None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:41.877101 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b05dfd1-95a7-4fc9-b55b-d7c71b2372c8 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:41.893294 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:41.897841 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 247/500] 10.222.0.18 () {58 vars in 1282 bytes} [Sat Aug 6 21:50:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 21:50:41.899103 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aee281e-62f5-4786-ac47-aeff9661f096 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:41.899387 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aee281e-62f5-4786-ac47-aeff9661f096 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:41.899662 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aee281e-62f5-4786-ac47-aeff9661f096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:41.900489 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:41.917020 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4e80cd-58c3-4c84-a918-7c017f398619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:41.917499 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-6aee281e-62f5-4786-ac47-aeff9661f096 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:50:41.917638 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4e80cd-58c3-4c84-a918-7c017f398619 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:41.917913 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4e80cd-58c3-4c84-a918-7c017f398619 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:41.918182 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4e80cd-58c3-4c84-a918-7c017f398619 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:41.918694 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-6aee281e-62f5-4786-ac47-aeff9661f096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:50:41.925167 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4e80cd-58c3-4c84-a918-7c017f398619 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:41.926627 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4e80cd-58c3-4c84-a918-7c017f398619 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:41.940836 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-df4e80cd-58c3-4c84-a918-7c017f398619 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 06 21:50:41.940836 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:41.940836 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:41.940836 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:41.940836 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:41.940836 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:41.940836 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:41.940836 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:41.940836 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:41.940836 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:41.940836 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:41.940836 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:41.940836 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 21:50:41.940836 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 21:50:41.940836 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 21:50:41.940836 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 21:50:41.941746 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:41.941746 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:41.941746 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 21:50:41.941746 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 21:50:41.941746 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 21:50:41.941746 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 21:50:41.941746 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:41.941746 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:41.941746 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:41.941746 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:41.941746 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:41.941746 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:41.941746 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:41.941746 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:41.941746 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:41.941746 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:41.941746 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:41.941746 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:41.942625 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:41.942625 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:41.942625 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 21:50:41.942625 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 21:50:41.942625 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 21:50:41.942625 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 21:50:41.942625 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 21:50:41.942625 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 21:50:41.942625 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 06 21:50:41.942625 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:50:41.942625 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 254/501] 10.222.0.18 () {56 vars in 1099 bytes} [Sat Aug 6 21:50:41 2022] GET /identity/v3/users/glance => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:41.943189 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e644430e-d3d8-46fb-a8e2-08d86a146856 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:41.943315 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e644430e-d3d8-46fb-a8e2-08d86a146856 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:41.943624 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e644430e-d3d8-46fb-a8e2-08d86a146856 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:41.945132 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 255/502] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:50:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:50:41.945509 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e644430e-d3d8-46fb-a8e2-08d86a146856 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:41.986067 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 248/503] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:50:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 21:50:41.986905 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aee281e-62f5-4786-ac47-aeff9661f096 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:42.050630 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e644430e-d3d8-46fb-a8e2-08d86a146856 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:42.083660 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c06579f-2fdf-4e2f-95b1-d4003cdd554f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:42.084279 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c06579f-2fdf-4e2f-95b1-d4003cdd554f None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:42.084592 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c06579f-2fdf-4e2f-95b1-d4003cdd554f None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:42.084864 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c06579f-2fdf-4e2f-95b1-d4003cdd554f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:42.095785 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c06579f-2fdf-4e2f-95b1-d4003cdd554f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:42.097282 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c06579f-2fdf-4e2f-95b1-d4003cdd554f None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:42.097705 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aee281e-62f5-4786-ac47-aeff9661f096 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:42.111453 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-0c06579f-2fdf-4e2f-95b1-d4003cdd554f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:50:42.111453 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:42.111453 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:42.111453 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:42.111453 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:42.111453 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:42.111453 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:42.111453 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:42.111453 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:42.111453 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:42.111453 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:42.111453 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:42.111453 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:50:42.111453 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:50:42.111453 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:50:42.111453 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:50:42.112517 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:42.112517 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:42.112517 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:42.112517 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:42.112517 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:42.112517 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:42.112517 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:42.112517 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:42.112517 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:42.112517 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:42.112517 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:42.112517 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:42.112517 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:42.112517 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:42.112517 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:42.112517 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:42.112517 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:50:42.113400 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:50:42.113400 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:50:42.113400 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:50:42.113400 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 256/504] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:50:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:42.113400 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c06579f-2fdf-4e2f-95b1-d4003cdd554f None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:42.127991 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:42.135683 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c53a5a7-0e89-4048-9e1c-cc04ce508d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:42.136309 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c53a5a7-0e89-4048-9e1c-cc04ce508d68 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:42.136616 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c53a5a7-0e89-4048-9e1c-cc04ce508d68 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:42.136887 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c53a5a7-0e89-4048-9e1c-cc04ce508d68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:42.143835 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:42.146973 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c53a5a7-0e89-4048-9e1c-cc04ce508d68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:42.148431 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c53a5a7-0e89-4048-9e1c-cc04ce508d68 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:42.163553 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-2c53a5a7-0e89-4048-9e1c-cc04ce508d68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:50:42.163553 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:42.163553 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:42.163553 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:42.163553 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:42.163553 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:42.163553 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:42.163553 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:42.163553 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:42.163553 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:42.163553 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:42.163553 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:42.163553 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:50:42.163553 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:50:42.163553 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:50:42.163553 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:50:42.164697 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:42.164697 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:42.164697 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:42.164697 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:42.164697 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:42.164697 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:42.164697 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:42.164697 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:42.164697 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:42.164697 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:42.164697 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:42.164697 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:42.164697 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:42.164697 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:42.164697 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:42.164697 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:42.164697 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:50:42.165789 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:50:42.165789 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:50:42.165789 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:50:42.165789 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 249/505] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:50:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:42.166207 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036c318b-6679-401d-81d4-0b3f4d90931d None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:42.166480 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036c318b-6679-401d-81d4-0b3f4d90931d None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:42.166747 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036c318b-6679-401d-81d4-0b3f4d90931d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:42.185334 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-036c318b-6679-401d-81d4-0b3f4d90931d None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:50:42.186513 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-036c318b-6679-401d-81d4-0b3f4d90931d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:50:42.217083 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c06579f-2fdf-4e2f-95b1-d4003cdd554f None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:42.250215 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24024134-0e43-4771-b9dc-c8f0017e3f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:42.250963 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24024134-0e43-4771-b9dc-c8f0017e3f63 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:42.251273 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24024134-0e43-4771-b9dc-c8f0017e3f63 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:42.251542 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24024134-0e43-4771-b9dc-c8f0017e3f63 None admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:42.252266 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24024134-0e43-4771-b9dc-c8f0017e3f63 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:42.252596 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24024134-0e43-4771-b9dc-c8f0017e3f63 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:42.254032 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24024134-0e43-4771-b9dc-c8f0017e3f63 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:42.254346 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 250/506] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:50:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 21:50:42.255177 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036c318b-6679-401d-81d4-0b3f4d90931d None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:42.278786 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 257/507] 10.222.0.18 () {56 vars in 1137 bytes} [Sat Aug 6 21:50:42 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:42.279679 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24024134-0e43-4771-b9dc-c8f0017e3f63 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:42.355853 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036c318b-6679-401d-81d4-0b3f4d90931d None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:42.366703 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:42.368288 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:42.378565 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24024134-0e43-4771-b9dc-c8f0017e3f63 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:42.390922 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1f2db7-08b9-4c6a-9e36-e1ef6e522535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:42.392167 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1f2db7-08b9-4c6a-9e36-e1ef6e522535 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:42.392547 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1f2db7-08b9-4c6a-9e36-e1ef6e522535 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:42.392820 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1f2db7-08b9-4c6a-9e36-e1ef6e522535 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:42.403628 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1f2db7-08b9-4c6a-9e36-e1ef6e522535 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:50:42.408553 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-da1f2db7-08b9-4c6a-9e36-e1ef6e522535 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 21:50:42.408553 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:42.408553 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:42.408553 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:42.408553 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:42.408553 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:42.408553 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:42.408553 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:42.408553 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:42.408553 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:42.408553 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:42.408553 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:42.408553 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:50:42.408553 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:50:42.408553 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:50:42.408553 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:50:42.409564 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:50:42.409564 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:50:42.409564 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:50:42.409564 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:50:42.409564 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:42.409564 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:42.409564 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:42.409564 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:42.409564 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:42.409564 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:42.409564 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:42.409564 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:42.409564 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:42.409564 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:42.409564 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:42.409564 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:42.410364 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:42.410364 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:42.410364 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:42.410364 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:42.410364 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:50:42.410364 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:50:42.410364 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:50:42.410364 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:50:42.410364 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:50:42.410364 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:50:42.410364 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 21:50:42.410364 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:50:42.410364 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 251/508] 10.222.0.18 () {56 vars in 1102 bytes} [Sat Aug 6 21:50:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:42.411209 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1f2db7-08b9-4c6a-9e36-e1ef6e522535 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:42.416858 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e938d5-8603-49e7-82b9-9a0116ac23af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:42.417461 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e938d5-8603-49e7-82b9-9a0116ac23af None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:42.417732 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e938d5-8603-49e7-82b9-9a0116ac23af None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:42.417999 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e938d5-8603-49e7-82b9-9a0116ac23af None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:42.418638 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e938d5-8603-49e7-82b9-9a0116ac23af None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:42.418935 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e938d5-8603-49e7-82b9-9a0116ac23af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:42.420322 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e938d5-8603-49e7-82b9-9a0116ac23af None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:42.432242 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 258/509] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:50:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:42.433093 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e938d5-8603-49e7-82b9-9a0116ac23af None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:42.517769 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1f2db7-08b9-4c6a-9e36-e1ef6e522535 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:42.529902 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e938d5-8603-49e7-82b9-9a0116ac23af None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:42.551200 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b4d5df-7ffc-4f8e-8488-df637d1fd0b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:42.551798 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b4d5df-7ffc-4f8e-8488-df637d1fd0b8 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:42.552076 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b4d5df-7ffc-4f8e-8488-df637d1fd0b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:42.552348 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b4d5df-7ffc-4f8e-8488-df637d1fd0b8 None admin] PATH_INFO: `/v3/domains` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:42.553032 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b4d5df-7ffc-4f8e-8488-df637d1fd0b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:42.553335 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b4d5df-7ffc-4f8e-8488-df637d1fd0b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:42.554707 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b4d5df-7ffc-4f8e-8488-df637d1fd0b8 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:42.564180 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acae498e-f153-454b-8d94-b37fbfe92627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:42.564919 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae498e-f153-454b-8d94-b37fbfe92627 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:42.565197 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae498e-f153-454b-8d94-b37fbfe92627 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:42.565464 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae498e-f153-454b-8d94-b37fbfe92627 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:42.566184 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 252/510] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:50:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:42.566995 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b4d5df-7ffc-4f8e-8488-df637d1fd0b8 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:42.576652 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acae498e-f153-454b-8d94-b37fbfe92627 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:42.578152 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acae498e-f153-454b-8d94-b37fbfe92627 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:42.586913 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:42.593278 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-acae498e-f153-454b-8d94-b37fbfe92627 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:50:42.593278 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:42.593278 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:42.593278 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:42.593278 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:42.593278 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:42.593278 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:42.593278 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:42.593278 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:42.593278 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:42.593278 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:42.593278 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:42.593278 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:50:42.593278 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:50:42.593278 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:50:42.593278 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:50:42.601248 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:42.601248 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:42.601248 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:42.601248 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:42.601248 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:42.601248 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:42.601248 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:42.601248 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:42.601248 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:42.601248 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:42.601248 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:42.601248 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:42.601248 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:42.601248 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:42.601248 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:42.601248 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:42.601248 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:50:42.602319 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:50:42.602319 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:50:42.602319 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:50:42.602319 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 259/511] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:50:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:42.602319 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acae498e-f153-454b-8d94-b37fbfe92627 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:42.602594 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:42.667636 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b4d5df-7ffc-4f8e-8488-df637d1fd0b8 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:42.698559 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acae498e-f153-454b-8d94-b37fbfe92627 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:42.700862 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5423a48f-e4d6-4e37-8f43-a796da427549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:42.701468 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5423a48f-e4d6-4e37-8f43-a796da427549 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:42.701744 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5423a48f-e4d6-4e37-8f43-a796da427549 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:42.702013 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5423a48f-e4d6-4e37-8f43-a796da427549 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:42.731477 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649999dc-c050-48b1-90dd-e14544e0d9c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:42.732068 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649999dc-c050-48b1-90dd-e14544e0d9c9 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:42.732627 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649999dc-c050-48b1-90dd-e14544e0d9c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:42.732897 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649999dc-c050-48b1-90dd-e14544e0d9c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:42.733547 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649999dc-c050-48b1-90dd-e14544e0d9c9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:42.733844 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649999dc-c050-48b1-90dd-e14544e0d9c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:42.735882 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649999dc-c050-48b1-90dd-e14544e0d9c9 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:42.744413 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 260/512] 10.222.0.18 () {56 vars in 1103 bytes} [Sat Aug 6 21:50:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:42.745228 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649999dc-c050-48b1-90dd-e14544e0d9c9 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:42.772165 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5423a48f-e4d6-4e37-8f43-a796da427549 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:42.773749 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5423a48f-e4d6-4e37-8f43-a796da427549 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:42.802545 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:42.804480 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:42.842389 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649999dc-c050-48b1-90dd-e14544e0d9c9 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:42.873053 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 253/513] 10.222.0.18 () {58 vars in 1282 bytes} [Sat Aug 6 21:50:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 21:50:42.873828 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5423a48f-e4d6-4e37-8f43-a796da427549 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:42.875222 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c432ded7-3a98-413b-8e94-c6aa5f2efc4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:42.876426 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c432ded7-3a98-413b-8e94-c6aa5f2efc4a None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:42.876777 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c432ded7-3a98-413b-8e94-c6aa5f2efc4a None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:42.877048 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c432ded7-3a98-413b-8e94-c6aa5f2efc4a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:42.884956 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c432ded7-3a98-413b-8e94-c6aa5f2efc4a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:42.886423 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c432ded7-3a98-413b-8e94-c6aa5f2efc4a None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:42.897884 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-c432ded7-3a98-413b-8e94-c6aa5f2efc4a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 06 21:50:42.897884 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:42.897884 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:42.897884 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:42.897884 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:42.897884 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:42.897884 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:42.897884 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:42.897884 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:42.897884 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:42.897884 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:42.897884 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:42.897884 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 21:50:42.897884 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 21:50:42.897884 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 21:50:42.897884 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 21:50:42.899003 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:42.899003 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:42.899003 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 21:50:42.899003 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 21:50:42.899003 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 21:50:42.899003 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 21:50:42.899003 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:42.899003 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:42.899003 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:42.899003 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:42.899003 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:42.899003 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:42.899003 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:42.899003 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:42.899003 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:42.899003 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:42.899003 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:42.899003 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:42.899946 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:42.899946 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:42.899946 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 21:50:42.899946 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 21:50:42.899946 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 21:50:42.899946 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 21:50:42.899946 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 21:50:42.899946 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 21:50:42.899946 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 06 21:50:42.899946 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:50:42.899946 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 261/514] 10.222.0.18 () {56 vars in 1099 bytes} [Sat Aug 6 21:50:42 2022] GET /identity/v3/users/cinder => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:42.900545 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c432ded7-3a98-413b-8e94-c6aa5f2efc4a None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:42.976224 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5423a48f-e4d6-4e37-8f43-a796da427549 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:42.998265 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c432ded7-3a98-413b-8e94-c6aa5f2efc4a None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:43.004504 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:43.010311 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13140369-9729-4546-a7bd-cc874b5b1a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:43.010934 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13140369-9729-4546-a7bd-cc874b5b1a82 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:43.011209 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13140369-9729-4546-a7bd-cc874b5b1a82 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:43.011485 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13140369-9729-4546-a7bd-cc874b5b1a82 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:43.018556 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13140369-9729-4546-a7bd-cc874b5b1a82 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:43.020006 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13140369-9729-4546-a7bd-cc874b5b1a82 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:43.021204 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:43.031740 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-13140369-9729-4546-a7bd-cc874b5b1a82 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 21:50:43.031740 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:43.031740 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:43.031740 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:43.031740 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:43.031740 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:43.031740 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:43.031740 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:43.031740 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:43.031740 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:43.031740 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:43.031740 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:43.031740 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 21:50:43.031740 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 21:50:43.031740 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 21:50:43.033059 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 21:50:43.033059 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:43.033059 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:43.033059 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:43.033059 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:43.033059 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:43.033059 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:43.033059 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:43.033059 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:43.033059 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:43.033059 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:43.033059 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:43.033059 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:43.033059 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:43.033059 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:43.033059 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:43.033059 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:43.033899 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 21:50:43.033899 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 21:50:43.033899 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 21:50:43.033899 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 21:50:43.033899 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 21:50:43.033899 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 21:50:43.033899 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 21:50:43.033899 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:50:43.034323 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5734964-012b-4142-af4c-7c3721364b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:43.034323 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5734964-012b-4142-af4c-7c3721364b32 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:43.034496 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 254/515] 10.222.0.18 () {56 vars in 1111 bytes} [Sat Aug 6 21:50:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:43.034561 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5734964-012b-4142-af4c-7c3721364b32 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:43.034561 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5734964-012b-4142-af4c-7c3721364b32 None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:43.034677 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13140369-9729-4546-a7bd-cc874b5b1a82 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:43.034752 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5734964-012b-4142-af4c-7c3721364b32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:43.034820 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5734964-012b-4142-af4c-7c3721364b32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:43.036161 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5734964-012b-4142-af4c-7c3721364b32 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:43.048590 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 262/516] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:50:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:43.049474 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5734964-012b-4142-af4c-7c3721364b32 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:43.143291 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13140369-9729-4546-a7bd-cc874b5b1a82 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:43.149251 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5734964-012b-4142-af4c-7c3721364b32 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:43.176687 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4274d7-bf2e-4d95-af7a-a1b45084c8ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:43.177310 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4274d7-bf2e-4d95-af7a-a1b45084c8ab None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:43.177587 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4274d7-bf2e-4d95-af7a-a1b45084c8ab None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:43.177858 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4274d7-bf2e-4d95-af7a-a1b45084c8ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:43.182055 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39032e10-d025-4764-84ee-cf3d5e172430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:43.182668 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39032e10-d025-4764-84ee-cf3d5e172430 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:43.182945 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39032e10-d025-4764-84ee-cf3d5e172430 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:43.183215 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39032e10-d025-4764-84ee-cf3d5e172430 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:43.188026 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4274d7-bf2e-4d95-af7a-a1b45084c8ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:43.189995 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4274d7-bf2e-4d95-af7a-a1b45084c8ab None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:43.193402 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39032e10-d025-4764-84ee-cf3d5e172430 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:43.194834 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39032e10-d025-4764-84ee-cf3d5e172430 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:43.204963 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-de4274d7-bf2e-4d95-af7a-a1b45084c8ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:50:43.204963 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:43.204963 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:43.204963 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:43.204963 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:43.204963 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:43.204963 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:43.204963 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:43.204963 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:43.204963 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:43.204963 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:43.204963 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:43.204963 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:50:43.204963 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:50:43.204963 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:50:43.204963 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:50:43.205873 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:43.205873 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:43.205873 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:43.205873 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:43.205873 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:43.205873 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:43.205873 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:43.205873 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:43.205873 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:43.205873 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:43.205873 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:43.205873 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:43.205873 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:43.205873 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:43.205873 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:43.205873 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:43.205873 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:50:43.206724 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:50:43.206724 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:50:43.206724 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:50:43.206724 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 255/517] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:50:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:43.206937 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4274d7-bf2e-4d95-af7a-a1b45084c8ab None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:43.209371 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-39032e10-d025-4764-84ee-cf3d5e172430 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:50:43.209371 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:43.209371 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:43.209371 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:43.209371 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:43.209371 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:43.209371 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:43.209371 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:43.209371 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:43.209371 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:43.209371 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:43.209371 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:43.209371 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:50:43.209371 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:50:43.209371 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:50:43.209371 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:50:43.210397 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:43.210397 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:43.210397 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:43.210397 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:43.210397 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:43.210397 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:43.210397 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:43.210397 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:43.210397 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:43.210397 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:43.210397 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:43.210397 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:43.210397 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:43.210397 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:43.210397 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:43.210397 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:43.210397 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:50:43.215892 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:50:43.215892 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:50:43.215892 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:50:43.215892 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 263/518] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:50:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:43.215892 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39032e10-d025-4764-84ee-cf3d5e172430 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:43.246752 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:43.254024 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:43.308604 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4274d7-bf2e-4d95-af7a-a1b45084c8ab None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:43.311710 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39032e10-d025-4764-84ee-cf3d5e172430 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:43.342693 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311cbf6b-cbe8-4ece-bab3-9c3bd2e0f2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:43.343284 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311cbf6b-cbe8-4ece-bab3-9c3bd2e0f2ef None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:43.343560 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311cbf6b-cbe8-4ece-bab3-9c3bd2e0f2ef None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:43.343831 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311cbf6b-cbe8-4ece-bab3-9c3bd2e0f2ef None admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:43.344615 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311cbf6b-cbe8-4ece-bab3-9c3bd2e0f2ef None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:43.345020 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311cbf6b-cbe8-4ece-bab3-9c3bd2e0f2ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:43.345683 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571bcbb8-2bbe-4d26-99f5-85e7ff2319ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:43.346275 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571bcbb8-2bbe-4d26-99f5-85e7ff2319ae None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:43.346475 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311cbf6b-cbe8-4ece-bab3-9c3bd2e0f2ef None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:43.346569 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571bcbb8-2bbe-4d26-99f5-85e7ff2319ae None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:43.346803 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571bcbb8-2bbe-4d26-99f5-85e7ff2319ae None admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:43.347482 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571bcbb8-2bbe-4d26-99f5-85e7ff2319ae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:43.347780 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571bcbb8-2bbe-4d26-99f5-85e7ff2319ae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:43.349243 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571bcbb8-2bbe-4d26-99f5-85e7ff2319ae None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:43.367100 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 264/519] 10.222.0.18 () {56 vars in 1148 bytes} [Sat Aug 6 21:50:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:43.367912 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571bcbb8-2bbe-4d26-99f5-85e7ff2319ae None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:43.385900 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 256/520] 10.222.0.18 () {56 vars in 1137 bytes} [Sat Aug 6 21:50:43 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:43.386742 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311cbf6b-cbe8-4ece-bab3-9c3bd2e0f2ef None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:43.467030 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:43.473605 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571bcbb8-2bbe-4d26-99f5-85e7ff2319ae None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:43.483019 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:43.485948 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311cbf6b-cbe8-4ece-bab3-9c3bd2e0f2ef None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:43.507701 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ddca15-b8a9-418f-b2af-af1b3c19cbd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:43.508305 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ddca15-b8a9-418f-b2af-af1b3c19cbd8 None admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:43.508609 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ddca15-b8a9-418f-b2af-af1b3c19cbd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:43.508878 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ddca15-b8a9-418f-b2af-af1b3c19cbd8 None admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:43.509554 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ddca15-b8a9-418f-b2af-af1b3c19cbd8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:43.510988 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ddca15-b8a9-418f-b2af-af1b3c19cbd8 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:43.520487 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69655da6-8531-43e6-be29-d71c3f1c324e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:43.521252 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69655da6-8531-43e6-be29-d71c3f1c324e None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:43.521861 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69655da6-8531-43e6-be29-d71c3f1c324e None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:43.522137 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69655da6-8531-43e6-be29-d71c3f1c324e None admin] PATH_INFO: `/v3/domains` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:43.522774 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69655da6-8531-43e6-be29-d71c3f1c324e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:43.523070 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69655da6-8531-43e6-be29-d71c3f1c324e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:43.524445 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69655da6-8531-43e6-be29-d71c3f1c324e None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:43.536383 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 257/521] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:50:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:43.537272 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69655da6-8531-43e6-be29-d71c3f1c324e None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:43.612856 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 265/522] 10.222.0.18 () {60 vars in 1132 bytes} [Sat Aug 6 21:50:43 2022] POST /identity/v3/users => generated 247 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 21:50:43.613742 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ddca15-b8a9-418f-b2af-af1b3c19cbd8 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:43.642855 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69655da6-8531-43e6-be29-d71c3f1c324e None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:43.683882 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499db55f-991c-4057-b556-9f1d266adf6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:43.684607 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499db55f-991c-4057-b556-9f1d266adf6e None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:43.684955 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499db55f-991c-4057-b556-9f1d266adf6e None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:43.685704 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499db55f-991c-4057-b556-9f1d266adf6e None admin] PATH_INFO: `/v3/domains` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:43.686392 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499db55f-991c-4057-b556-9f1d266adf6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:43.686867 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499db55f-991c-4057-b556-9f1d266adf6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:43.688368 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499db55f-991c-4057-b556-9f1d266adf6e None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:43.703626 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 258/523] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:50:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:43.706591 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499db55f-991c-4057-b556-9f1d266adf6e None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:43.731384 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:43.742599 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ddca15-b8a9-418f-b2af-af1b3c19cbd8 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:43.781260 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9013741-b4c7-4200-bd04-f2083fb9c7f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:43.781260 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9013741-b4c7-4200-bd04-f2083fb9c7f6 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:43.781551 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9013741-b4c7-4200-bd04-f2083fb9c7f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:43.781812 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9013741-b4c7-4200-bd04-f2083fb9c7f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:43.782581 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9013741-b4c7-4200-bd04-f2083fb9c7f6 None admin] RBAC: Adding query filter params (role.id=d90649d4ef604eeb8edcf0aa22932ca1, scope.project.id=4bb6307eb0644d85a1d1daaa61352697, user.id=2a1319d33e9144859c6010843be312a3) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:43.782883 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9013741-b4c7-4200-bd04-f2083fb9c7f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:43.784307 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9013741-b4c7-4200-bd04-f2083fb9c7f6 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:43.789953 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 266/524] 10.222.0.18 () {56 vars in 1374 bytes} [Sat Aug 6 21:50:43 2022] GET /identity/v3/role_assignments?user.id=2a1319d33e9144859c6010843be312a3&scope.project.id=4bb6307eb0644d85a1d1daaa61352697&role.id=d90649d4ef604eeb8edcf0aa22932ca1 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:43.790814 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9013741-b4c7-4200-bd04-f2083fb9c7f6 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:43.809194 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:43.830456 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499db55f-991c-4057-b556-9f1d266adf6e None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:43.880482 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355d5d75-0a1c-491a-aae1-d6c1ca9c03a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:43.881216 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d5d75-0a1c-491a-aae1-d6c1ca9c03a3 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:43.881562 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d5d75-0a1c-491a-aae1-d6c1ca9c03a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:43.881839 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d5d75-0a1c-491a-aae1-d6c1ca9c03a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:43.922177 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9013741-b4c7-4200-bd04-f2083fb9c7f6 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:43.963561 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9284211-7ac1-4604-8538-7d51b310a28e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:43.964313 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9284211-7ac1-4604-8538-7d51b310a28e None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:43.964622 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9284211-7ac1-4604-8538-7d51b310a28e None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:43.964895 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9284211-7ac1-4604-8538-7d51b310a28e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:43.967660 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355d5d75-0a1c-491a-aae1-d6c1ca9c03a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:43.969577 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355d5d75-0a1c-491a-aae1-d6c1ca9c03a3 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:43.975775 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9284211-7ac1-4604-8538-7d51b310a28e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:43.977285 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9284211-7ac1-4604-8538-7d51b310a28e None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:43.990249 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-c9284211-7ac1-4604-8538-7d51b310a28e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 06 21:50:43.990249 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:43.990249 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:43.990249 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:43.990249 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:43.990249 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:43.990249 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:43.990249 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:43.990249 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:43.990249 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:43.990249 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:43.990249 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:43.990249 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 21:50:43.990249 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 21:50:43.990249 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 21:50:43.990249 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 21:50:43.991191 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:43.991191 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:43.991191 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 21:50:43.991191 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 21:50:43.991191 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 21:50:43.991191 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 21:50:43.991191 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:43.991191 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:43.991191 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:43.991191 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:43.991191 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:43.991191 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:43.991191 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:43.991191 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:43.991191 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:43.991191 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:43.991191 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:43.991191 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:43.992069 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:43.992069 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:43.992069 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 21:50:43.992069 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 21:50:43.992069 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 21:50:43.992069 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 21:50:43.992069 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 21:50:43.992069 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 21:50:43.992069 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 06 21:50:43.992069 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:50:43.992069 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 267/525] 10.222.0.18 () {56 vars in 1102 bytes} [Sat Aug 6 21:50:43 2022] GET /identity/v3/users/neutron => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:43.992069 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9284211-7ac1-4604-8538-7d51b310a28e None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:44.000153 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:44.077396 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:44.103835 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9284211-7ac1-4604-8538-7d51b310a28e None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:44.109688 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 259/526] 10.222.0.18 () {58 vars in 1282 bytes} [Sat Aug 6 21:50:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 21:50:44.111020 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355d5d75-0a1c-491a-aae1-d6c1ca9c03a3 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:44.153026 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0f2e8e-fd6d-42bb-9f38-dbfd1e87dafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:44.153680 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0f2e8e-fd6d-42bb-9f38-dbfd1e87dafe None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:44.153923 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0f2e8e-fd6d-42bb-9f38-dbfd1e87dafe None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:44.154188 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0f2e8e-fd6d-42bb-9f38-dbfd1e87dafe None admin] PATH_INFO: `/v3/projects/service` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:44.164672 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0f2e8e-fd6d-42bb-9f38-dbfd1e87dafe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:44.167576 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0f2e8e-fd6d-42bb-9f38-dbfd1e87dafe None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:44.184665 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-4e0f2e8e-fd6d-42bb-9f38-dbfd1e87dafe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 21:50:44.184665 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:44.184665 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:44.184665 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:44.184665 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:44.184665 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:44.184665 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:44.184665 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:44.184665 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:44.184665 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:44.184665 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:44.184665 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:44.184665 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 21:50:44.184665 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 21:50:44.184665 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 21:50:44.185550 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 21:50:44.185550 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:44.185550 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:44.185550 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:44.185550 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:44.185550 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:44.185550 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:44.185550 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:44.185550 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:44.185550 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:44.185550 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:44.185550 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:44.185550 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:44.185550 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:44.185550 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:44.185550 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:44.185550 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:44.186434 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 21:50:44.186434 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 21:50:44.186434 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 21:50:44.186434 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 21:50:44.186434 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 21:50:44.186434 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 21:50:44.186434 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 21:50:44.186434 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:50:44.189027 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 268/527] 10.222.0.18 () {56 vars in 1111 bytes} [Sat Aug 6 21:50:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:44.189969 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0f2e8e-fd6d-42bb-9f38-dbfd1e87dafe None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:44.251789 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355d5d75-0a1c-491a-aae1-d6c1ca9c03a3 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:44.279358 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:44.305565 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcea9966-5244-4fd4-ad53-5cd31309b4d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:44.306865 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcea9966-5244-4fd4-ad53-5cd31309b4d7 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:44.306865 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcea9966-5244-4fd4-ad53-5cd31309b4d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:44.306865 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcea9966-5244-4fd4-ad53-5cd31309b4d7 None admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:44.307636 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcea9966-5244-4fd4-ad53-5cd31309b4d7 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:44.312435 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcea9966-5244-4fd4-ad53-5cd31309b4d7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:44.312435 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcea9966-5244-4fd4-ad53-5cd31309b4d7 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:44.323466 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0f2e8e-fd6d-42bb-9f38-dbfd1e87dafe None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:44.330302 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 260/528] 10.222.0.18 () {56 vars in 1139 bytes} [Sat Aug 6 21:50:44 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:44.331180 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcea9966-5244-4fd4-ad53-5cd31309b4d7 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:44.369335 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb8ca5f-96a3-4143-ad7e-b78276d55718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:44.369983 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb8ca5f-96a3-4143-ad7e-b78276d55718 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:44.370258 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb8ca5f-96a3-4143-ad7e-b78276d55718 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:44.370523 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb8ca5f-96a3-4143-ad7e-b78276d55718 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:44.372449 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:44.382600 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb8ca5f-96a3-4143-ad7e-b78276d55718 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:44.384067 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb8ca5f-96a3-4143-ad7e-b78276d55718 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:44.398918 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-0cb8ca5f-96a3-4143-ad7e-b78276d55718 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:50:44.398918 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:44.398918 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:44.398918 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:44.398918 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:44.398918 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:44.398918 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:44.398918 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:44.398918 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:44.398918 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:44.398918 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:44.398918 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:44.398918 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:50:44.398918 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:50:44.398918 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:50:44.398918 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:50:44.400107 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:44.400107 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:44.400107 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:44.400107 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:44.400107 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:44.400107 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:44.400107 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:44.400107 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:44.400107 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:44.400107 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:44.400107 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:44.400107 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:44.400107 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:44.400107 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:44.400107 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:44.400107 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:44.400107 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:50:44.401185 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:50:44.401185 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:50:44.401185 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:50:44.401185 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 269/529] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:50:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:44.401185 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb8ca5f-96a3-4143-ad7e-b78276d55718 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:44.447334 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcea9966-5244-4fd4-ad53-5cd31309b4d7 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:44.486601 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776fb737-ada4-48d2-8153-081d630b2a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:44.487502 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776fb737-ada4-48d2-8153-081d630b2a76 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:44.488018 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776fb737-ada4-48d2-8153-081d630b2a76 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:44.488479 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776fb737-ada4-48d2-8153-081d630b2a76 None admin] PATH_INFO: `/v3/projects` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:44.489390 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776fb737-ada4-48d2-8153-081d630b2a76 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:44.489839 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776fb737-ada4-48d2-8153-081d630b2a76 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:44.491675 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776fb737-ada4-48d2-8153-081d630b2a76 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:44.506695 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb8ca5f-96a3-4143-ad7e-b78276d55718 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:44.511827 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 261/530] 10.222.0.18 () {56 vars in 1148 bytes} [Sat Aug 6 21:50:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:44.512857 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776fb737-ada4-48d2-8153-081d630b2a76 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:44.547159 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cb8e17-4092-42b5-9caf-607a8eb5e041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:44.548663 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cb8e17-4092-42b5-9caf-607a8eb5e041 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:44.548833 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cb8e17-4092-42b5-9caf-607a8eb5e041 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:44.549199 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cb8e17-4092-42b5-9caf-607a8eb5e041 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:44.558647 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:44.606508 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:44.620466 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cb8e17-4092-42b5-9caf-607a8eb5e041 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:44.622007 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cb8e17-4092-42b5-9caf-607a8eb5e041 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:44.628117 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776fb737-ada4-48d2-8153-081d630b2a76 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:44.666581 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d109ae-0ff7-4d17-9833-97d8eda380fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:44.667454 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d109ae-0ff7-4d17-9833-97d8eda380fc None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:44.667930 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d109ae-0ff7-4d17-9833-97d8eda380fc None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:44.668346 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d109ae-0ff7-4d17-9833-97d8eda380fc None admin] PATH_INFO: `/v3/domains` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:44.669577 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d109ae-0ff7-4d17-9833-97d8eda380fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:44.670036 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d109ae-0ff7-4d17-9833-97d8eda380fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:44.672033 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d109ae-0ff7-4d17-9833-97d8eda380fc None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:44.686195 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 262/531] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:50:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:44.687243 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d109ae-0ff7-4d17-9833-97d8eda380fc None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:44.724240 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 270/532] 10.222.0.18 () {58 vars in 1282 bytes} [Sat Aug 6 21:50:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 21:50:44.729354 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cb8e17-4092-42b5-9caf-607a8eb5e041 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:44.796368 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:44.801561 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d109ae-0ff7-4d17-9833-97d8eda380fc None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:44.820790 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:44.831871 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cb8e17-4092-42b5-9caf-607a8eb5e041 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:44.846939 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d00241-3c2b-4c65-b194-ebff484bd13f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:44.848166 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d00241-3c2b-4c65-b194-ebff484bd13f None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:44.849218 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d00241-3c2b-4c65-b194-ebff484bd13f None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:44.849681 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d00241-3c2b-4c65-b194-ebff484bd13f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:44.851049 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d00241-3c2b-4c65-b194-ebff484bd13f None admin] RBAC: Adding query filter params (role.id=d90649d4ef604eeb8edcf0aa22932ca1, scope.project.id=4bb6307eb0644d85a1d1daaa61352697, user.id=c8d38e73e105476e8de858cfa939ba20) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:44.851554 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d00241-3c2b-4c65-b194-ebff484bd13f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:44.853682 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d00241-3c2b-4c65-b194-ebff484bd13f None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:44.862840 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 263/533] 10.222.0.18 () {56 vars in 1374 bytes} [Sat Aug 6 21:50:44 2022] GET /identity/v3/role_assignments?user.id=c8d38e73e105476e8de858cfa939ba20&scope.project.id=4bb6307eb0644d85a1d1daaa61352697&role.id=d90649d4ef604eeb8edcf0aa22932ca1 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:44.863878 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d00241-3c2b-4c65-b194-ebff484bd13f None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:44.929296 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e6e0c1-7e97-4b52-846b-0094e83060e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:44.929921 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e6e0c1-7e97-4b52-846b-0094e83060e0 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:44.930183 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e6e0c1-7e97-4b52-846b-0094e83060e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:44.930460 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e6e0c1-7e97-4b52-846b-0094e83060e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:44.953336 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e6e0c1-7e97-4b52-846b-0094e83060e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:44.954909 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e6e0c1-7e97-4b52-846b-0094e83060e0 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:44.978369 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-63e6e0c1-7e97-4b52-846b-0094e83060e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 21:50:44.978369 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:44.978369 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:44.978369 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:44.978369 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:44.978369 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:44.978369 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:44.978369 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:44.978369 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:44.978369 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:44.978369 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:44.978369 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:44.978369 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 21:50:44.978369 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 21:50:44.978369 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 21:50:44.979329 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 21:50:44.979329 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:44.979329 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:44.979329 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:44.979329 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:44.979329 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:44.979329 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:44.979329 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:44.979329 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:44.979329 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:44.979329 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:44.979329 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:44.979329 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:44.979329 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:44.979329 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:44.979329 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:44.979329 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:44.980192 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 21:50:44.980192 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 21:50:44.980192 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 21:50:44.980192 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 21:50:44.980192 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 21:50:44.980192 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 21:50:44.980192 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 21:50:44.980192 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:50:44.989020 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 271/534] 10.222.0.18 () {56 vars in 1111 bytes} [Sat Aug 6 21:50:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:44.991642 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d00241-3c2b-4c65-b194-ebff484bd13f None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:44.995670 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e6e0c1-7e97-4b52-846b-0094e83060e0 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:45.037901 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd712faa-6ecf-4c98-b81e-ab57056aa754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:45.039276 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd712faa-6ecf-4c98-b81e-ab57056aa754 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:45.039756 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd712faa-6ecf-4c98-b81e-ab57056aa754 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:45.040239 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd712faa-6ecf-4c98-b81e-ab57056aa754 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:45.053531 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd712faa-6ecf-4c98-b81e-ab57056aa754 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:45.055838 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd712faa-6ecf-4c98-b81e-ab57056aa754 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:45.074243 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-dd712faa-6ecf-4c98-b81e-ab57056aa754 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:50:45.074243 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:45.074243 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:45.074243 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:45.074243 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:45.074243 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:45.074243 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:45.074243 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:45.074243 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:45.074243 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:45.074243 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:45.074243 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:45.074243 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:50:45.074243 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:50:45.074243 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:50:45.074243 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:50:45.075217 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:45.075217 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:45.075217 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:45.075217 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:45.075217 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:45.075217 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:45.075217 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:45.075217 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:45.075217 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:45.075217 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:45.075217 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:45.075217 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:45.075217 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:45.075217 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:45.075217 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:45.075217 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:45.075217 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:50:45.076059 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:50:45.076059 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:50:45.076059 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:50:45.078244 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 264/535] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:50:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:45.084085 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd712faa-6ecf-4c98-b81e-ab57056aa754 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:45.104820 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:45.176140 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e6e0c1-7e97-4b52-846b-0094e83060e0 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:45.210988 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd712faa-6ecf-4c98-b81e-ab57056aa754 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:45.250258 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2545927d-454d-4306-9dca-d9ac407a30c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:45.252065 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2545927d-454d-4306-9dca-d9ac407a30c5 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:45.252696 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2545927d-454d-4306-9dca-d9ac407a30c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:45.253176 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2545927d-454d-4306-9dca-d9ac407a30c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:45.254185 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2545927d-454d-4306-9dca-d9ac407a30c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:45.254709 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2545927d-454d-4306-9dca-d9ac407a30c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:45.256300 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2545927d-454d-4306-9dca-d9ac407a30c5 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:45.270317 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 265/536] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:50:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:45.275608 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2545927d-454d-4306-9dca-d9ac407a30c5 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:45.281654 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95675d3b-bea9-41b7-9710-b83c02acb6c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:45.282642 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95675d3b-bea9-41b7-9710-b83c02acb6c2 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:45.285146 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95675d3b-bea9-41b7-9710-b83c02acb6c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:45.285634 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95675d3b-bea9-41b7-9710-b83c02acb6c2 None admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:45.289235 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95675d3b-bea9-41b7-9710-b83c02acb6c2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:45.289758 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95675d3b-bea9-41b7-9710-b83c02acb6c2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:45.293001 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95675d3b-bea9-41b7-9710-b83c02acb6c2 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:45.321676 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 272/537] 10.222.0.18 () {56 vars in 1148 bytes} [Sat Aug 6 21:50:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:50:45.323209 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95675d3b-bea9-41b7-9710-b83c02acb6c2 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:45.352790 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:45.444089 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:45.491187 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95675d3b-bea9-41b7-9710-b83c02acb6c2 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:45.530103 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2fa5ee-e0ff-43fd-8c9f-6189de032c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:45.530810 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2fa5ee-e0ff-43fd-8c9f-6189de032c22 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:45.531160 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2fa5ee-e0ff-43fd-8c9f-6189de032c22 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:45.531747 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2fa5ee-e0ff-43fd-8c9f-6189de032c22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:45.532664 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2fa5ee-e0ff-43fd-8c9f-6189de032c22 None admin] RBAC: Adding query filter params (role.id=d90649d4ef604eeb8edcf0aa22932ca1, scope.project.id=4bb6307eb0644d85a1d1daaa61352697, user.id=aa71c71de263478da234f3c5716a6e5f) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:45.533070 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2fa5ee-e0ff-43fd-8c9f-6189de032c22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:45.535176 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2fa5ee-e0ff-43fd-8c9f-6189de032c22 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:45.541550 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 273/538] 10.222.0.18 () {56 vars in 1374 bytes} [Sat Aug 6 21:50:45 2022] GET /identity/v3/role_assignments?user.id=aa71c71de263478da234f3c5716a6e5f&scope.project.id=4bb6307eb0644d85a1d1daaa61352697&role.id=d90649d4ef604eeb8edcf0aa22932ca1 => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:45.542411 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2545927d-454d-4306-9dca-d9ac407a30c5 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:45.627384 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e23aa3-dcfb-4897-b5bf-de1455f30a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:45.627955 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e23aa3-dcfb-4897-b5bf-de1455f30a12 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:45.628232 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e23aa3-dcfb-4897-b5bf-de1455f30a12 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:45.628522 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e23aa3-dcfb-4897-b5bf-de1455f30a12 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:45.641926 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e23aa3-dcfb-4897-b5bf-de1455f30a12 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:45.643413 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e23aa3-dcfb-4897-b5bf-de1455f30a12 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:45.666035 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-41e23aa3-dcfb-4897-b5bf-de1455f30a12 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 21:50:45.666035 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:45.666035 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:45.666035 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:45.666035 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:45.666035 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:45.666035 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:45.666035 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:45.666035 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:45.666035 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:45.666035 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:45.666035 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:45.666035 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 21:50:45.666035 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 21:50:45.666035 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 21:50:45.666974 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 21:50:45.666974 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:45.666974 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:45.666974 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:45.666974 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:45.666974 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:45.666974 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:45.666974 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:45.666974 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:45.666974 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:45.666974 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:45.666974 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:45.666974 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:45.666974 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:45.666974 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:45.666974 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:45.666974 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:45.667847 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 21:50:45.667847 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 21:50:45.667847 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 21:50:45.667847 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 21:50:45.667847 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 21:50:45.667847 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 21:50:45.667847 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 21:50:45.667847 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:50:45.667847 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 266/539] 10.222.0.18 () {56 vars in 1111 bytes} [Sat Aug 6 21:50:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 392 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:45.673182 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2fa5ee-e0ff-43fd-8c9f-6189de032c22 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:45.775656 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24565456-c42e-4d58-9631-821a729ee0cc None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:45.775972 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24565456-c42e-4d58-9631-821a729ee0cc None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:45.776205 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24565456-c42e-4d58-9631-821a729ee0cc None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:45.777824 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 267/540] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:50:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:50:45.783805 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdef5f2-aae8-4357-8880-0743d7755392 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:45.784101 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdef5f2-aae8-4357-8880-0743d7755392 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:45.784358 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdef5f2-aae8-4357-8880-0743d7755392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:45.805888 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:45.806241 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-fbdef5f2-aae8-4357-8880-0743d7755392 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:50:45.807451 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-fbdef5f2-aae8-4357-8880-0743d7755392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:50:45.865847 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2fa5ee-e0ff-43fd-8c9f-6189de032c22 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:45.913116 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 268/541] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:50:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 21:50:45.931764 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72146e3b-a47d-4d20-a5d1-811f07fb3f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:45.932375 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72146e3b-a47d-4d20-a5d1-811f07fb3f57 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:45.932677 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72146e3b-a47d-4d20-a5d1-811f07fb3f57 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:45.932961 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72146e3b-a47d-4d20-a5d1-811f07fb3f57 None admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:45.933636 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72146e3b-a47d-4d20-a5d1-811f07fb3f57 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:45.933957 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72146e3b-a47d-4d20-a5d1-811f07fb3f57 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:45.935432 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72146e3b-a47d-4d20-a5d1-811f07fb3f57 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:45.952877 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 274/542] 10.222.0.18 () {56 vars in 1148 bytes} [Sat Aug 6 21:50:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:45.959318 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdef5f2-aae8-4357-8880-0743d7755392 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:45.989729 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72146e3b-a47d-4d20-a5d1-811f07fb3f57 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:46.072345 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:46.114974 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdef5f2-aae8-4357-8880-0743d7755392 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:46.116973 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72146e3b-a47d-4d20-a5d1-811f07fb3f57 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:46.149783 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:46.155055 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab5c9ce-7be8-4baa-bab4-6571d360c3f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:46.155689 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab5c9ce-7be8-4baa-bab4-6571d360c3f9 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:46.155968 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab5c9ce-7be8-4baa-bab4-6571d360c3f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:46.156243 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab5c9ce-7be8-4baa-bab4-6571d360c3f9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:46.158544 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925f66ae-bd10-48b2-bfe3-227fabe8512d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:46.159683 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925f66ae-bd10-48b2-bfe3-227fabe8512d None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:46.160123 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925f66ae-bd10-48b2-bfe3-227fabe8512d None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:46.169240 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925f66ae-bd10-48b2-bfe3-227fabe8512d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:46.169240 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925f66ae-bd10-48b2-bfe3-227fabe8512d None admin] RBAC: Adding query filter params (role.id=d90649d4ef604eeb8edcf0aa22932ca1, scope.project.id=4bb6307eb0644d85a1d1daaa61352697, user.id=5ceaeadc843c4186b25114b28607fe22) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:46.169240 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925f66ae-bd10-48b2-bfe3-227fabe8512d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:46.169240 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925f66ae-bd10-48b2-bfe3-227fabe8512d None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:46.169523 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab5c9ce-7be8-4baa-bab4-6571d360c3f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:50:46.176480 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-8ab5c9ce-7be8-4baa-bab4-6571d360c3f9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 21:50:46.176480 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:46.176480 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:46.176480 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:46.176480 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:46.176480 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:46.176480 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:46.176480 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:46.176480 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:46.176480 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:46.176480 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:46.176480 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:46.176480 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:50:46.176480 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:50:46.176480 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:50:46.176480 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:50:46.177823 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:50:46.177823 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:50:46.177823 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:50:46.177823 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:50:46.177823 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:46.177823 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:46.177823 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:46.177823 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:46.177823 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:46.177823 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:46.177823 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:46.177823 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:46.177823 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:46.177823 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:46.177823 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:46.177823 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:46.178628 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:46.178628 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:46.178628 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:46.178628 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:46.178628 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:50:46.178628 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:50:46.178628 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:50:46.178628 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:50:46.178628 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:50:46.178628 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:50:46.178628 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 21:50:46.178628 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:50:46.178628 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 275/544] 10.222.0.18 () {56 vars in 1102 bytes} [Sat Aug 6 21:50:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 21:50:46.178628 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c7e7e2-4feb-4d23-b91d-70bcb6c4b1f0 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:46.179316 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c7e7e2-4feb-4d23-b91d-70bcb6c4b1f0 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:46.179316 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c7e7e2-4feb-4d23-b91d-70bcb6c4b1f0 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:46.179316 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 276/545] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:50:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:50:46.180121 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 269/545] 10.222.0.18 () {56 vars in 1374 bytes} [Sat Aug 6 21:50:45 2022] GET /identity/v3/role_assignments?user.id=5ceaeadc843c4186b25114b28607fe22&scope.project.id=4bb6307eb0644d85a1d1daaa61352697&role.id=d90649d4ef604eeb8edcf0aa22932ca1 => generated 259 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:46.183925 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda3521f-e80c-4fcc-a733-ff77d652c212 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:46.184203 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda3521f-e80c-4fcc-a733-ff77d652c212 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:46.186574 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda3521f-e80c-4fcc-a733-ff77d652c212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:46.196960 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925f66ae-bd10-48b2-bfe3-227fabe8512d None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:46.223202 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-eda3521f-e80c-4fcc-a733-ff77d652c212 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:50:46.224845 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-eda3521f-e80c-4fcc-a733-ff77d652c212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:50:46.374813 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 277/546] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:50:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 21:50:46.403598 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925f66ae-bd10-48b2-bfe3-227fabe8512d None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:46.428764 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda3521f-e80c-4fcc-a733-ff77d652c212 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:46.463193 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:46.478311 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf14bf8-9538-41d5-a3be-ad4a861fb477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:46.478936 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf14bf8-9538-41d5-a3be-ad4a861fb477 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:46.479216 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf14bf8-9538-41d5-a3be-ad4a861fb477 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:46.479490 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf14bf8-9538-41d5-a3be-ad4a861fb477 None admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:46.480179 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf14bf8-9538-41d5-a3be-ad4a861fb477 None admin] RBAC: Adding query filter params (name=service) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:46.480506 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf14bf8-9538-41d5-a3be-ad4a861fb477 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:46.490306 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf14bf8-9538-41d5-a3be-ad4a861fb477 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:46.505319 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 270/547] 10.222.0.18 () {56 vars in 1103 bytes} [Sat Aug 6 21:50:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:50:46.510907 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf14bf8-9538-41d5-a3be-ad4a861fb477 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:46.600227 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda3521f-e80c-4fcc-a733-ff77d652c212 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:46.624833 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:46.642144 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab74bca-7c7f-4333-b1d2-5e5930acce8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:46.643134 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab74bca-7c7f-4333-b1d2-5e5930acce8b None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:46.643621 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab74bca-7c7f-4333-b1d2-5e5930acce8b None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:46.644086 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab74bca-7c7f-4333-b1d2-5e5930acce8b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:46.656657 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab74bca-7c7f-4333-b1d2-5e5930acce8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:50:46.663297 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-2ab74bca-7c7f-4333-b1d2-5e5930acce8b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 21:50:46.663297 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:46.663297 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:46.663297 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:46.663297 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:46.663297 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:46.663297 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:46.663297 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:46.663297 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:46.663297 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:46.663297 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:46.663297 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:46.663297 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:50:46.663297 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:50:46.663297 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:50:46.663297 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:50:46.664259 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:50:46.664259 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:50:46.664259 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:50:46.664259 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:50:46.664259 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:46.664259 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:46.664259 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:46.664259 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:46.664259 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:46.664259 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:46.664259 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:46.664259 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:46.664259 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:46.664259 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:46.664259 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:46.664259 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:46.665083 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:46.665083 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:46.665083 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:46.665083 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:46.665083 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:50:46.665083 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:50:46.665083 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:50:46.665083 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:50:46.665083 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:50:46.665083 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:50:46.665083 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 21:50:46.665083 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:50:46.668256 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 278/548] 10.222.0.18 () {56 vars in 1102 bytes} [Sat Aug 6 21:50:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 21:50:46.674763 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab74bca-7c7f-4333-b1d2-5e5930acce8b None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:46.792573 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab74bca-7c7f-4333-b1d2-5e5930acce8b None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:46.797141 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf14bf8-9538-41d5-a3be-ad4a861fb477 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:46.833056 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec312d6-f038-47fa-9762-1b89860ef9f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:46.834100 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec312d6-f038-47fa-9762-1b89860ef9f4 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:46.834721 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec312d6-f038-47fa-9762-1b89860ef9f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:46.835208 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec312d6-f038-47fa-9762-1b89860ef9f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:46.836135 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec312d6-f038-47fa-9762-1b89860ef9f4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:46.836667 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec312d6-f038-47fa-9762-1b89860ef9f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:46.838366 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec312d6-f038-47fa-9762-1b89860ef9f4 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:46.849487 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 279/549] 10.222.0.18 () {56 vars in 1103 bytes} [Sat Aug 6 21:50:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:46.855696 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec312d6-f038-47fa-9762-1b89860ef9f4 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:46.903590 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55186453-eb7e-4349-826a-07449fb8f316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:46.904290 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55186453-eb7e-4349-826a-07449fb8f316 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:46.904599 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55186453-eb7e-4349-826a-07449fb8f316 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:46.904930 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55186453-eb7e-4349-826a-07449fb8f316 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:46.906122 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:46.927057 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55186453-eb7e-4349-826a-07449fb8f316 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:46.928554 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55186453-eb7e-4349-826a-07449fb8f316 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:46.962099 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-55186453-eb7e-4349-826a-07449fb8f316 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:50:46.962099 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:46.962099 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:46.962099 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:46.962099 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:46.962099 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:46.962099 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:46.962099 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:46.962099 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:46.962099 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:46.962099 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:46.962099 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:46.962099 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:50:46.962099 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:50:46.962099 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:50:46.962099 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:50:46.963086 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:46.963086 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:46.963086 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:46.963086 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:46.963086 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:46.963086 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:46.963086 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:46.963086 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:46.963086 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:46.963086 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:46.963086 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:46.963086 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:46.963086 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:46.963086 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:46.963086 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:46.963086 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:46.963086 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:50:46.963933 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:50:46.963933 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:50:46.963933 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:50:46.968965 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 271/550] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:50:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 458 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 21:50:46.976084 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55186453-eb7e-4349-826a-07449fb8f316 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:46.978608 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec312d6-f038-47fa-9762-1b89860ef9f4 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:47.019081 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a4a9c2-dfde-4381-86ee-a0a439802038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:47.020059 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a4a9c2-dfde-4381-86ee-a0a439802038 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:47.020703 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a4a9c2-dfde-4381-86ee-a0a439802038 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:47.021184 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a4a9c2-dfde-4381-86ee-a0a439802038 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:47.033366 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a4a9c2-dfde-4381-86ee-a0a439802038 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:47.035574 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a4a9c2-dfde-4381-86ee-a0a439802038 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:47.052231 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-f3a4a9c2-dfde-4381-86ee-a0a439802038 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:50:47.052231 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:47.052231 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:47.052231 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:47.052231 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:47.052231 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:47.052231 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:47.052231 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:47.052231 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:47.052231 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:47.052231 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:47.052231 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:47.052231 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:50:47.052231 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:50:47.052231 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:50:47.052231 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:50:47.053346 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:47.053346 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:47.053346 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:47.053346 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:47.053346 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:47.053346 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:47.053346 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:47.053346 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:47.053346 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:47.053346 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:47.053346 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:47.053346 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:47.053346 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:47.053346 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:47.053346 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:47.053346 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:47.053346 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:50:47.054208 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:50:47.054208 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:50:47.054208 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:50:47.056326 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 280/551] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:50:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:47.062512 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a4a9c2-dfde-4381-86ee-a0a439802038 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:47.084795 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:47.191236 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55186453-eb7e-4349-826a-07449fb8f316 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:47.226937 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c5cea9-e832-4b93-8758-38242c79f314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:47.227706 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c5cea9-e832-4b93-8758-38242c79f314 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:47.227824 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c5cea9-e832-4b93-8758-38242c79f314 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:47.228074 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c5cea9-e832-4b93-8758-38242c79f314 None admin] PATH_INFO: `/v3/domains` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:47.228759 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c5cea9-e832-4b93-8758-38242c79f314 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:47.229066 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c5cea9-e832-4b93-8758-38242c79f314 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:47.230493 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c5cea9-e832-4b93-8758-38242c79f314 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:47.237753 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a4a9c2-dfde-4381-86ee-a0a439802038 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:47.245829 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 272/552] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:50:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:47.250298 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6fca42-a4b2-4a41-8ef7-4e4c0a23fa7e None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:47.250298 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6fca42-a4b2-4a41-8ef7-4e4c0a23fa7e None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:47.250298 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6fca42-a4b2-4a41-8ef7-4e4c0a23fa7e None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:47.250298 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 273/553] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:50:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:50:47.251712 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6fca42-a4b2-4a41-8ef7-4e4c0a23fa7e None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:47.254016 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:47.283731 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fbbcba-514f-4bae-b07c-8c2e49b46234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:47.284792 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fbbcba-514f-4bae-b07c-8c2e49b46234 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:47.285347 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fbbcba-514f-4bae-b07c-8c2e49b46234 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:47.285852 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fbbcba-514f-4bae-b07c-8c2e49b46234 None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:47.286748 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fbbcba-514f-4bae-b07c-8c2e49b46234 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:47.287368 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fbbcba-514f-4bae-b07c-8c2e49b46234 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:47.289245 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fbbcba-514f-4bae-b07c-8c2e49b46234 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:47.304218 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 281/554] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:50:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:50:47.305923 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa38aec-30b6-4812-8925-fb2a93b50b68 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:47.306414 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa38aec-30b6-4812-8925-fb2a93b50b68 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:47.307015 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa38aec-30b6-4812-8925-fb2a93b50b68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:47.327045 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-7aa38aec-30b6-4812-8925-fb2a93b50b68 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:50:47.328798 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-7aa38aec-30b6-4812-8925-fb2a93b50b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:50:47.359725 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6fca42-a4b2-4a41-8ef7-4e4c0a23fa7e None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:47.362654 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:47.398853 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ae55b8-8cca-4217-afff-378a12ea8681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:47.399939 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae55b8-8cca-4217-afff-378a12ea8681 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:47.400449 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae55b8-8cca-4217-afff-378a12ea8681 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:47.400978 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae55b8-8cca-4217-afff-378a12ea8681 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:47.410388 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ae55b8-8cca-4217-afff-378a12ea8681 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:47.412346 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ae55b8-8cca-4217-afff-378a12ea8681 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:47.417355 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 282/555] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:50:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 21:50:47.418374 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa38aec-30b6-4812-8925-fb2a93b50b68 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:47.433906 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-b0ae55b8-8cca-4217-afff-378a12ea8681 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 06 21:50:47.433906 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:47.433906 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:47.433906 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:47.433906 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:47.433906 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:47.433906 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:47.433906 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:47.433906 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:47.433906 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:47.433906 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:47.433906 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:47.433906 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 21:50:47.433906 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 21:50:47.433906 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 21:50:47.433906 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 21:50:47.434912 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:47.434912 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:47.434912 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 21:50:47.434912 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 21:50:47.434912 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 21:50:47.434912 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 21:50:47.434912 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:47.434912 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:47.434912 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:47.434912 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:47.434912 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:47.434912 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:47.434912 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:47.434912 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:47.434912 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:47.434912 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:47.434912 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:47.434912 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:47.435835 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:47.435835 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:47.435835 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 21:50:47.435835 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 21:50:47.435835 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 21:50:47.435835 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 21:50:47.435835 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 21:50:47.435835 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 21:50:47.435835 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 06 21:50:47.435835 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:50:47.439922 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 274/556] 10.222.0.18 () {56 vars in 1096 bytes} [Sat Aug 6 21:50:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:47.447360 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ae55b8-8cca-4217-afff-378a12ea8681 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:47.546643 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:47.565983 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ae55b8-8cca-4217-afff-378a12ea8681 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:47.569773 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa38aec-30b6-4812-8925-fb2a93b50b68 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:47.599320 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d19348f-8ae4-41cb-a764-c9d5c4d811e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:47.600544 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d19348f-8ae4-41cb-a764-c9d5c4d811e4 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:47.601188 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d19348f-8ae4-41cb-a764-c9d5c4d811e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:47.601678 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d19348f-8ae4-41cb-a764-c9d5c4d811e4 None admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:47.603168 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d19348f-8ae4-41cb-a764-c9d5c4d811e4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:47.603686 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d19348f-8ae4-41cb-a764-c9d5c4d811e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:47.605969 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d19348f-8ae4-41cb-a764-c9d5c4d811e4 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:47.617127 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09f7c11-2802-4d00-84e4-4324e5f40b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:47.618281 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f7c11-2802-4d00-84e4-4324e5f40b24 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:47.618742 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f7c11-2802-4d00-84e4-4324e5f40b24 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:47.619377 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f7c11-2802-4d00-84e4-4324e5f40b24 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:47.628803 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09f7c11-2802-4d00-84e4-4324e5f40b24 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:47.629792 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 275/557] 10.222.0.18 () {56 vars in 1135 bytes} [Sat Aug 6 21:50:47 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:47.630772 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d19348f-8ae4-41cb-a764-c9d5c4d811e4 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:47.631105 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09f7c11-2802-4d00-84e4-4324e5f40b24 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:47.646661 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-e09f7c11-2802-4d00-84e4-4324e5f40b24 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 06 21:50:47.646661 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:47.646661 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:47.646661 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:47.646661 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:47.646661 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:47.646661 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:47.646661 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:47.646661 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:47.646661 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:47.646661 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:47.646661 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:47.646661 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 21:50:47.646661 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 21:50:47.646661 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 21:50:47.646661 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 21:50:47.648067 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:47.648067 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:47.648067 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 21:50:47.648067 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 21:50:47.648067 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 21:50:47.648067 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 21:50:47.648067 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:47.648067 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:47.648067 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:47.648067 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:47.648067 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:47.648067 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:47.648067 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:47.648067 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:47.648067 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:47.648067 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:47.648067 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:47.648067 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:47.649036 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:47.649036 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:47.649036 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 21:50:47.649036 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 21:50:47.649036 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 21:50:47.649036 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 21:50:47.649036 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 21:50:47.649036 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 21:50:47.649036 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 06 21:50:47.649036 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:50:47.650280 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 283/558] 10.222.0.18 () {56 vars in 1093 bytes} [Sat Aug 6 21:50:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:47.650722 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09f7c11-2802-4d00-84e4-4324e5f40b24 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:47.659820 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:47.761973 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09f7c11-2802-4d00-84e4-4324e5f40b24 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:47.764307 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d19348f-8ae4-41cb-a764-c9d5c4d811e4 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:47.801981 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074b6935-1e54-4abb-ac0e-d0f43f922c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:47.803030 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074b6935-1e54-4abb-ac0e-d0f43f922c9c None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:47.803587 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074b6935-1e54-4abb-ac0e-d0f43f922c9c None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:47.804072 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074b6935-1e54-4abb-ac0e-d0f43f922c9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:47.809380 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b51600-b5ee-4161-828c-f4ae83b240ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:47.810024 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b51600-b5ee-4161-828c-f4ae83b240ef None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:47.810303 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b51600-b5ee-4161-828c-f4ae83b240ef None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:47.810575 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b51600-b5ee-4161-828c-f4ae83b240ef None admin] PATH_INFO: `/v3/roles/service` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:47.822867 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:47.823754 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b51600-b5ee-4161-828c-f4ae83b240ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:50:47.830254 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-72b51600-b5ee-4161-828c-f4ae83b240ef None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 21:50:47.830254 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:47.830254 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:47.830254 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:47.830254 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:47.830254 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:47.830254 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:47.830254 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:47.830254 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:47.830254 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:47.830254 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:47.830254 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:47.830254 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:50:47.830254 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:50:47.830254 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:50:47.830254 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:50:47.837704 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:50:47.837704 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:50:47.837704 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:50:47.837704 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:50:47.837704 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:47.837704 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:47.837704 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:47.837704 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:47.837704 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:47.837704 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:47.837704 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:47.837704 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:47.837704 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:47.837704 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:47.837704 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:47.837704 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:47.838555 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:47.838555 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:47.838555 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:47.838555 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:47.838555 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:50:47.838555 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:50:47.838555 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:50:47.838555 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:50:47.838555 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:50:47.838555 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:50:47.838555 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 21:50:47.838555 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:50:47.838555 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 276/559] 10.222.0.18 () {56 vars in 1102 bytes} [Sat Aug 6 21:50:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:47.838555 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b51600-b5ee-4161-828c-f4ae83b240ef None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:47.894183 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074b6935-1e54-4abb-ac0e-d0f43f922c9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:47.896075 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074b6935-1e54-4abb-ac0e-d0f43f922c9c None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:47.927031 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:47.948130 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b51600-b5ee-4161-828c-f4ae83b240ef None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:47.987030 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f1447b-c293-4a23-98d8-80f82c167077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:47.987924 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f1447b-c293-4a23-98d8-80f82c167077 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:47.988389 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f1447b-c293-4a23-98d8-80f82c167077 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:47.988902 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f1447b-c293-4a23-98d8-80f82c167077 None admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:47.989835 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f1447b-c293-4a23-98d8-80f82c167077 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:47.990223 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f1447b-c293-4a23-98d8-80f82c167077 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:47.991656 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f1447b-c293-4a23-98d8-80f82c167077 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:48.013717 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 277/560] 10.222.0.18 () {56 vars in 1133 bytes} [Sat Aug 6 21:50:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:48.013717 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f1447b-c293-4a23-98d8-80f82c167077 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:48.013990 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 284/561] 10.222.0.18 () {58 vars in 1282 bytes} [Sat Aug 6 21:50:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 21:50:48.013990 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7499e1-d562-4fb1-b9fb-dbc7ba2bcb21 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:48.013990 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7499e1-d562-4fb1-b9fb-dbc7ba2bcb21 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:48.013990 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7499e1-d562-4fb1-b9fb-dbc7ba2bcb21 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:48.018620 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 285/562] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:50:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:50:48.018620 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7499e1-d562-4fb1-b9fb-dbc7ba2bcb21 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:48.069898 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:48.123989 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f1447b-c293-4a23-98d8-80f82c167077 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:48.129199 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7499e1-d562-4fb1-b9fb-dbc7ba2bcb21 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:48.162947 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52563869-cb8e-42ff-af44-6744e1fa8388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:48.163555 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52563869-cb8e-42ff-af44-6744e1fa8388 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:48.164054 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52563869-cb8e-42ff-af44-6744e1fa8388 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:48.164371 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52563869-cb8e-42ff-af44-6744e1fa8388 None admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:48.165000 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fafde71-bd2e-439d-b5ff-1c26cb65c2d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:48.165126 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52563869-cb8e-42ff-af44-6744e1fa8388 None admin] RBAC: Adding query filter params (name=service) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:48.165325 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52563869-cb8e-42ff-af44-6744e1fa8388 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:48.165625 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fafde71-bd2e-439d-b5ff-1c26cb65c2d9 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:48.165900 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fafde71-bd2e-439d-b5ff-1c26cb65c2d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:48.166168 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fafde71-bd2e-439d-b5ff-1c26cb65c2d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:48.166738 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52563869-cb8e-42ff-af44-6744e1fa8388 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:48.171795 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:48.175623 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 278/563] 10.222.0.18 () {56 vars in 1103 bytes} [Sat Aug 6 21:50:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:48.178053 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fafde71-bd2e-439d-b5ff-1c26cb65c2d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:48.179504 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fafde71-bd2e-439d-b5ff-1c26cb65c2d9 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:48.181350 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52563869-cb8e-42ff-af44-6744e1fa8388 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:48.195569 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-7fafde71-bd2e-439d-b5ff-1c26cb65c2d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:50:48.195569 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:48.195569 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:48.195569 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:48.195569 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:48.195569 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:48.195569 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:48.195569 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:48.195569 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:48.195569 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:48.195569 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:48.195569 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:48.195569 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:50:48.195569 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:50:48.195569 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:50:48.195569 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:50:48.196814 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:48.196814 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:48.196814 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:48.196814 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:48.196814 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:48.196814 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:48.196814 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:48.196814 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:48.196814 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:48.196814 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:48.196814 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:48.196814 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:48.196814 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:48.196814 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:48.196814 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:48.196814 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:48.196814 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:50:48.197725 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:50:48.197725 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:50:48.197725 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:50:48.197725 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 286/564] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:50:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:48.197944 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3804f867-eaa4-4e7e-9caf-520962636bc2 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:48.198479 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3804f867-eaa4-4e7e-9caf-520962636bc2 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:48.198479 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3804f867-eaa4-4e7e-9caf-520962636bc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:48.215867 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-3804f867-eaa4-4e7e-9caf-520962636bc2 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:50:48.217117 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-3804f867-eaa4-4e7e-9caf-520962636bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:50:48.295385 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52563869-cb8e-42ff-af44-6744e1fa8388 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:48.303038 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 287/565] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:50:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 21:50:48.305062 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3804f867-eaa4-4e7e-9caf-520962636bc2 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:48.330972 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c89c4ab-bb5a-4969-965d-b459feb26e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:48.331596 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c89c4ab-bb5a-4969-965d-b459feb26e95 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:48.331873 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c89c4ab-bb5a-4969-965d-b459feb26e95 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:48.332144 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c89c4ab-bb5a-4969-965d-b459feb26e95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:48.341152 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:48.343077 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c89c4ab-bb5a-4969-965d-b459feb26e95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:48.345100 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c89c4ab-bb5a-4969-965d-b459feb26e95 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:48.361099 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-3c89c4ab-bb5a-4969-965d-b459feb26e95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:50:48.361099 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:48.361099 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:48.361099 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:48.361099 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:48.361099 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:48.361099 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:48.361099 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:48.361099 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:48.361099 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:48.361099 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:48.361099 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:48.361099 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:50:48.361099 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:50:48.361099 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:50:48.361099 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:50:48.362034 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:48.362034 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:48.362034 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:48.362034 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:48.362034 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:48.362034 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:48.362034 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:48.362034 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:48.362034 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:48.362034 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:48.362034 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:48.362034 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:48.362034 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:48.362034 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:48.362034 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:48.362034 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:48.362034 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:50:48.362903 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:50:48.362903 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:50:48.362903 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:50:48.362903 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 279/566] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:50:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:48.363125 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c89c4ab-bb5a-4969-965d-b459feb26e95 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:48.415880 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:48.424810 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3804f867-eaa4-4e7e-9caf-520962636bc2 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:48.459156 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c038143d-bca2-4aad-ae3f-56c91e369bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:48.459781 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c038143d-bca2-4aad-ae3f-56c91e369bf2 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:48.460060 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c038143d-bca2-4aad-ae3f-56c91e369bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:48.460328 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c038143d-bca2-4aad-ae3f-56c91e369bf2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:48.464108 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c89c4ab-bb5a-4969-965d-b459feb26e95 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:48.471046 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c038143d-bca2-4aad-ae3f-56c91e369bf2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:48.472540 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c038143d-bca2-4aad-ae3f-56c91e369bf2 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:48.487669 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-c038143d-bca2-4aad-ae3f-56c91e369bf2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:50:48.487669 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:48.487669 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:48.487669 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:48.487669 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:48.487669 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:48.487669 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:48.487669 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:48.487669 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:48.487669 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:48.487669 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:48.487669 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:48.487669 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:50:48.487669 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:50:48.487669 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:50:48.487669 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:50:48.488657 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:48.488657 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:48.488657 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:48.488657 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:48.488657 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:48.488657 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:48.488657 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:48.488657 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:48.488657 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:48.488657 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:48.488657 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:48.488657 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:48.488657 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:48.488657 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:48.488657 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:48.488657 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:48.488657 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:50:48.489551 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:50:48.489551 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:50:48.489551 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:50:48.489551 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 288/567] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:50:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:48.489787 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c038143d-bca2-4aad-ae3f-56c91e369bf2 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:48.501132 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb8a76e-f744-415b-b8fe-35feea813c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:48.501729 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb8a76e-f744-415b-b8fe-35feea813c8c None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:48.502004 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb8a76e-f744-415b-b8fe-35feea813c8c None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:48.502272 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb8a76e-f744-415b-b8fe-35feea813c8c None admin] PATH_INFO: `/v3/domains` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:48.502963 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb8a76e-f744-415b-b8fe-35feea813c8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:48.503260 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb8a76e-f744-415b-b8fe-35feea813c8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:48.504646 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb8a76e-f744-415b-b8fe-35feea813c8c None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:48.516682 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 280/568] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:50:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:48.522438 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d59b5d9-a250-4192-be79-159b70e234fc None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:48.522438 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d59b5d9-a250-4192-be79-159b70e234fc None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:48.522438 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d59b5d9-a250-4192-be79-159b70e234fc None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:48.522438 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 281/569] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:50:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:50:48.522438 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d59b5d9-a250-4192-be79-159b70e234fc None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:48.577125 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:48.598525 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c038143d-bca2-4aad-ae3f-56c91e369bf2 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:48.623551 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d59b5d9-a250-4192-be79-159b70e234fc None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:48.632332 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc30a54-ae9f-4c2d-8e21-a52da56855fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:48.632980 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc30a54-ae9f-4c2d-8e21-a52da56855fb None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:48.633258 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc30a54-ae9f-4c2d-8e21-a52da56855fb None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:48.633528 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc30a54-ae9f-4c2d-8e21-a52da56855fb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:48.636606 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:48.643821 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc30a54-ae9f-4c2d-8e21-a52da56855fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:50:48.648924 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-7dc30a54-ae9f-4c2d-8e21-a52da56855fb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 21:50:48.648924 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:48.648924 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:48.648924 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:48.648924 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:48.648924 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:48.648924 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:48.648924 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:48.648924 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:48.648924 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:48.648924 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:48.648924 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:48.648924 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:50:48.648924 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:50:48.648924 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:50:48.648924 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:50:48.654998 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:50:48.654998 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:50:48.654998 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:50:48.654998 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:50:48.654998 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:48.654998 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:48.654998 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:48.654998 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:48.654998 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:48.654998 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:48.654998 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:48.654998 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:48.654998 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:48.654998 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:48.654998 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:48.654998 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:48.656164 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:48.656164 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:48.656164 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:48.656164 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:48.656164 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:50:48.656164 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:50:48.656164 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:50:48.656164 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:50:48.656164 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:50:48.656164 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:50:48.656164 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 21:50:48.656164 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:50:48.656164 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 289/570] 10.222.0.18 () {56 vars in 1102 bytes} [Sat Aug 6 21:50:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:48.656164 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc30a54-ae9f-4c2d-8e21-a52da56855fb None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:48.658457 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0934a6-bc69-470c-9acd-8c90ceb26b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:48.659063 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0934a6-bc69-470c-9acd-8c90ceb26b7a None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:48.659343 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0934a6-bc69-470c-9acd-8c90ceb26b7a None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:48.659612 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0934a6-bc69-470c-9acd-8c90ceb26b7a None admin] PATH_INFO: `/v3/domains` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:48.660250 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0934a6-bc69-470c-9acd-8c90ceb26b7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:48.660580 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0934a6-bc69-470c-9acd-8c90ceb26b7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:48.661939 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0934a6-bc69-470c-9acd-8c90ceb26b7a None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:48.674299 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 282/571] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:50:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:48.675083 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0934a6-bc69-470c-9acd-8c90ceb26b7a None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:48.759202 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc30a54-ae9f-4c2d-8e21-a52da56855fb None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:48.772016 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0934a6-bc69-470c-9acd-8c90ceb26b7a None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:48.792741 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf18762-cad4-4ca8-a0d5-d22168d2efd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:48.793336 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf18762-cad4-4ca8-a0d5-d22168d2efd6 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:48.793617 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf18762-cad4-4ca8-a0d5-d22168d2efd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:48.793888 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf18762-cad4-4ca8-a0d5-d22168d2efd6 None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:48.794529 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf18762-cad4-4ca8-a0d5-d22168d2efd6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:48.794826 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf18762-cad4-4ca8-a0d5-d22168d2efd6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:48.796189 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf18762-cad4-4ca8-a0d5-d22168d2efd6 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:48.803016 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:48.804664 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e25661-7f1d-4a47-9728-16e017b3722f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:48.805276 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e25661-7f1d-4a47-9728-16e017b3722f None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:48.805555 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e25661-7f1d-4a47-9728-16e017b3722f None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:48.805820 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e25661-7f1d-4a47-9728-16e017b3722f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:48.808127 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 290/572] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:50:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:48.809304 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28189ddf-9e78-4db2-a58d-564cd4f8c048 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:48.809584 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28189ddf-9e78-4db2-a58d-564cd4f8c048 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:48.809854 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28189ddf-9e78-4db2-a58d-564cd4f8c048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:48.812797 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e25661-7f1d-4a47-9728-16e017b3722f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:48.815010 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e25661-7f1d-4a47-9728-16e017b3722f None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:48.826985 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-66e25661-7f1d-4a47-9728-16e017b3722f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 21:50:48.826985 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:48.826985 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:48.826985 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:48.826985 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:48.826985 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:48.826985 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:48.826985 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:48.826985 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:48.826985 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:48.826985 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:48.826985 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:48.826985 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 21:50:48.826985 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 21:50:48.826985 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 21:50:48.828050 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 21:50:48.828050 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:48.828050 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:48.828050 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:48.828050 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:48.828050 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:48.828050 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:48.828050 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:48.828050 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:48.828050 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:48.828050 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:48.828050 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:48.828050 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:48.828050 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:48.828050 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:48.828050 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:48.828050 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:48.834657 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 21:50:48.834657 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 21:50:48.834657 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 21:50:48.834657 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 21:50:48.834657 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 21:50:48.834657 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 21:50:48.834657 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 21:50:48.834657 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:50:48.835215 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-28189ddf-9e78-4db2-a58d-564cd4f8c048 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:50:48.835303 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 283/573] 10.222.0.18 () {56 vars in 1111 bytes} [Sat Aug 6 21:50:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:48.835303 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e25661-7f1d-4a47-9728-16e017b3722f None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:48.835452 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-28189ddf-9e78-4db2-a58d-564cd4f8c048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:50:48.859575 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:48.897276 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 291/574] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:50:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 21:50:48.898114 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28189ddf-9e78-4db2-a58d-564cd4f8c048 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:48.930889 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e25661-7f1d-4a47-9728-16e017b3722f None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:48.966120 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2190bb-9a61-4600-9fcc-2d32f6483079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:48.966718 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2190bb-9a61-4600-9fcc-2d32f6483079 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:48.966996 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2190bb-9a61-4600-9fcc-2d32f6483079 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:48.967293 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2190bb-9a61-4600-9fcc-2d32f6483079 None admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:48.967950 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2190bb-9a61-4600-9fcc-2d32f6483079 None admin] RBAC: Adding query filter params (name=service) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:48.968267 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2190bb-9a61-4600-9fcc-2d32f6483079 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:48.969860 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2190bb-9a61-4600-9fcc-2d32f6483079 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:48.977620 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 284/575] 10.222.0.18 () {56 vars in 1103 bytes} [Sat Aug 6 21:50:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:48.978393 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2190bb-9a61-4600-9fcc-2d32f6483079 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:48.994978 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28189ddf-9e78-4db2-a58d-564cd4f8c048 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:49.022149 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:49.029524 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06fe0c1-d72d-4157-a14c-8b3812a03a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:49.030144 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06fe0c1-d72d-4157-a14c-8b3812a03a4d None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:49.031730 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06fe0c1-d72d-4157-a14c-8b3812a03a4d None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:49.031730 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06fe0c1-d72d-4157-a14c-8b3812a03a4d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:49.037463 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06fe0c1-d72d-4157-a14c-8b3812a03a4d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:49.038898 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06fe0c1-d72d-4157-a14c-8b3812a03a4d None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:49.050033 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-a06fe0c1-d72d-4157-a14c-8b3812a03a4d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 21:50:49.050033 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:49.050033 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:49.050033 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:49.050033 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:49.050033 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:49.050033 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:49.050033 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:49.050033 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:49.050033 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:49.050033 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:49.050033 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:49.050033 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 21:50:49.050033 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 21:50:49.050033 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 21:50:49.051080 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 21:50:49.051080 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:49.051080 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:49.051080 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:49.051080 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:49.051080 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:49.051080 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:49.051080 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:49.051080 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:49.051080 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:49.051080 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:49.051080 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:49.051080 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:49.051080 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:49.051080 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:49.051080 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:49.051080 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:49.051990 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 21:50:49.051990 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 21:50:49.051990 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 21:50:49.051990 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 21:50:49.051990 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 21:50:49.051990 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 21:50:49.051990 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 21:50:49.051990 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:50:49.051990 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 292/576] 10.222.0.18 () {56 vars in 1111 bytes} [Sat Aug 6 21:50:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:49.051990 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06fe0c1-d72d-4157-a14c-8b3812a03a4d None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:49.074170 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:49.081229 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2190bb-9a61-4600-9fcc-2d32f6483079 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:49.115427 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643b1d83-2951-44d7-8039-e5dd2009244f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:49.116042 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b1d83-2951-44d7-8039-e5dd2009244f None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:49.116320 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b1d83-2951-44d7-8039-e5dd2009244f None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:49.117673 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b1d83-2951-44d7-8039-e5dd2009244f None admin] PATH_INFO: `/v3/users/glance` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:49.124494 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643b1d83-2951-44d7-8039-e5dd2009244f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:49.126117 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643b1d83-2951-44d7-8039-e5dd2009244f None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:49.137137 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-643b1d83-2951-44d7-8039-e5dd2009244f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 06 21:50:49.137137 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:49.137137 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:49.137137 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:49.137137 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:49.137137 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:49.137137 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:49.137137 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:49.137137 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:49.137137 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:49.137137 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:49.137137 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:49.137137 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 21:50:49.137137 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 21:50:49.137137 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 21:50:49.137137 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 21:50:49.138211 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:49.138211 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:49.138211 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 21:50:49.138211 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 21:50:49.138211 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 21:50:49.138211 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 21:50:49.138211 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:49.138211 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:49.138211 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:49.138211 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:49.138211 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:49.138211 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:49.138211 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:49.138211 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:49.138211 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:49.138211 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:49.138211 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:49.138211 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:49.139350 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:49.139350 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:49.139350 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 21:50:49.139350 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 21:50:49.139350 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 21:50:49.139350 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 21:50:49.139350 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 21:50:49.139350 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 21:50:49.139350 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 06 21:50:49.139350 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:50:49.139350 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 285/577] 10.222.0.18 () {56 vars in 1099 bytes} [Sat Aug 6 21:50:48 2022] GET /identity/v3/users/glance => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:49.139350 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643b1d83-2951-44d7-8039-e5dd2009244f None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:49.173264 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06fe0c1-d72d-4157-a14c-8b3812a03a4d None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:49.208111 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc22272-cecc-447b-9e33-5dae355dadc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:49.208740 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc22272-cecc-447b-9e33-5dae355dadc6 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:49.209017 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc22272-cecc-447b-9e33-5dae355dadc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:49.209287 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc22272-cecc-447b-9e33-5dae355dadc6 None admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:49.209965 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc22272-cecc-447b-9e33-5dae355dadc6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:49.210268 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc22272-cecc-447b-9e33-5dae355dadc6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:49.212096 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc22272-cecc-447b-9e33-5dae355dadc6 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:49.224707 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 293/578] 10.222.0.18 () {56 vars in 1148 bytes} [Sat Aug 6 21:50:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:49.225602 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc22272-cecc-447b-9e33-5dae355dadc6 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:49.237446 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643b1d83-2951-44d7-8039-e5dd2009244f None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:49.270959 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7879d6fb-86ae-4678-b048-cfbcc30a738d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:49.271668 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7879d6fb-86ae-4678-b048-cfbcc30a738d None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:49.271951 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7879d6fb-86ae-4678-b048-cfbcc30a738d None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:49.272224 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7879d6fb-86ae-4678-b048-cfbcc30a738d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:49.273463 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:49.282515 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7879d6fb-86ae-4678-b048-cfbcc30a738d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:50:49.287622 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-7879d6fb-86ae-4678-b048-cfbcc30a738d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 21:50:49.287622 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:49.287622 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:49.287622 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:49.287622 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:49.287622 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:49.287622 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:49.287622 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:49.287622 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:49.287622 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:49.287622 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:49.287622 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:49.287622 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:50:49.287622 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:50:49.287622 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:50:49.287622 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:50:49.288742 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:50:49.288742 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:50:49.288742 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:50:49.288742 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:50:49.288742 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:49.288742 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:49.288742 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:49.288742 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:49.288742 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:49.288742 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:49.288742 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:49.288742 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:49.288742 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:49.288742 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:49.288742 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:49.288742 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:49.289606 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:49.289606 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:49.289606 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:49.289606 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:49.289606 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:50:49.289606 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:50:49.289606 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:50:49.289606 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:50:49.289606 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:50:49.289606 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:50:49.289606 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 21:50:49.289606 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:50:49.289606 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 286/579] 10.222.0.18 () {56 vars in 1102 bytes} [Sat Aug 6 21:50:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:49.290299 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7879d6fb-86ae-4678-b048-cfbcc30a738d None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:49.307006 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:49.327503 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc22272-cecc-447b-9e33-5dae355dadc6 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:49.363598 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cce7a9a-0e88-449c-9930-76d5b7663fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:49.364228 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce7a9a-0e88-449c-9930-76d5b7663fd6 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:49.364540 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce7a9a-0e88-449c-9930-76d5b7663fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:49.364816 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce7a9a-0e88-449c-9930-76d5b7663fd6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:49.375283 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cce7a9a-0e88-449c-9930-76d5b7663fd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:49.377637 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cce7a9a-0e88-449c-9930-76d5b7663fd6 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:49.392460 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-2cce7a9a-0e88-449c-9930-76d5b7663fd6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:50:49.392460 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:49.392460 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:49.392460 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:49.392460 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:49.392460 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:49.392460 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:49.392460 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:49.392460 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:49.392460 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:49.392460 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:49.392460 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:49.392460 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:50:49.392460 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:50:49.392460 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:50:49.392460 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:50:49.393664 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:49.393664 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:49.393664 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:49.393664 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:49.393664 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:49.393664 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:49.393664 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:49.393664 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:49.393664 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:49.393664 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:49.393664 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:49.393664 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:49.393664 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:49.393664 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:49.393664 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:49.393664 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:49.393664 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:50:49.394754 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:50:49.394754 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:50:49.394754 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:50:49.394754 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 294/580] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:50:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:49.394754 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cce7a9a-0e88-449c-9930-76d5b7663fd6 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:49.398383 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7879d6fb-86ae-4678-b048-cfbcc30a738d None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:49.431852 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6b4a38-3b33-404d-acec-25848d0cda6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:49.432489 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6b4a38-3b33-404d-acec-25848d0cda6e None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:49.432773 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6b4a38-3b33-404d-acec-25848d0cda6e None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:49.433050 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6b4a38-3b33-404d-acec-25848d0cda6e None admin] PATH_INFO: `/v3/projects` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:49.433744 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6b4a38-3b33-404d-acec-25848d0cda6e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:49.434047 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6b4a38-3b33-404d-acec-25848d0cda6e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:49.435469 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6b4a38-3b33-404d-acec-25848d0cda6e None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:49.448645 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 287/581] 10.222.0.18 () {56 vars in 1148 bytes} [Sat Aug 6 21:50:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:49.449411 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6b4a38-3b33-404d-acec-25848d0cda6e None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:49.495339 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:49.498508 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cce7a9a-0e88-449c-9930-76d5b7663fd6 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:49.523854 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:49.532498 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4c700c-6ea0-41ca-91bb-27c913e731ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:49.533124 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c700c-6ea0-41ca-91bb-27c913e731ea None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:49.533404 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c700c-6ea0-41ca-91bb-27c913e731ea None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:49.533696 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c700c-6ea0-41ca-91bb-27c913e731ea None admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:49.534425 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4c700c-6ea0-41ca-91bb-27c913e731ea None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:49.534730 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4c700c-6ea0-41ca-91bb-27c913e731ea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:49.536175 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4c700c-6ea0-41ca-91bb-27c913e731ea None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:49.549649 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6b4a38-3b33-404d-acec-25848d0cda6e None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:49.553300 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 295/582] 10.222.0.18 () {56 vars in 1137 bytes} [Sat Aug 6 21:50:49 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:49.554120 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4c700c-6ea0-41ca-91bb-27c913e731ea None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:49.585205 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f52c25-5263-446d-9c79-d3d1a420f241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:49.585820 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f52c25-5263-446d-9c79-d3d1a420f241 None admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:49.586099 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f52c25-5263-446d-9c79-d3d1a420f241 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:49.586380 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f52c25-5263-446d-9c79-d3d1a420f241 None admin] PATH_INFO: `/v3/projects/4bb6307eb0644d85a1d1daaa61352697/users/2a1319d33e9144859c6010843be312a3/roles/d90649d4ef604eeb8edcf0aa22932ca1` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:49.600534 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f52c25-5263-446d-9c79-d3d1a420f241 None admin] RBAC: Authorizing `identity:create_grant(project_id=4bb6307eb0644d85a1d1daaa61352697, user_id=2a1319d33e9144859c6010843be312a3, role_id=d90649d4ef604eeb8edcf0aa22932ca1)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:49.601964 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f52c25-5263-446d-9c79-d3d1a420f241 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:49.653847 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4c700c-6ea0-41ca-91bb-27c913e731ea None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:49.669425 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 288/583] 10.222.0.18 () {58 vars in 1439 bytes} [Sat Aug 6 21:50:49 2022] PUT /identity/v3/projects/4bb6307eb0644d85a1d1daaa61352697/users/2a1319d33e9144859c6010843be312a3/roles/d90649d4ef604eeb8edcf0aa22932ca1 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 21:50:49.670253 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f52c25-5263-446d-9c79-d3d1a420f241 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:49.700098 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746a2db0-0f75-477a-aa19-63e3313bb6d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:49.700796 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746a2db0-0f75-477a-aa19-63e3313bb6d8 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:49.701078 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746a2db0-0f75-477a-aa19-63e3313bb6d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:49.701364 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746a2db0-0f75-477a-aa19-63e3313bb6d8 None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:49.702076 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746a2db0-0f75-477a-aa19-63e3313bb6d8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:49.702382 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746a2db0-0f75-477a-aa19-63e3313bb6d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:49.704785 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746a2db0-0f75-477a-aa19-63e3313bb6d8 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:49.713981 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 296/584] 10.222.0.18 () {56 vars in 1103 bytes} [Sat Aug 6 21:50:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:49.714852 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746a2db0-0f75-477a-aa19-63e3313bb6d8 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:49.741711 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:49.789506 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f52c25-5263-446d-9c79-d3d1a420f241 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:49.816732 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:49.832342 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746a2db0-0f75-477a-aa19-63e3313bb6d8 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:49.832557 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0eea15-15c2-464e-b4b7-45aead0afab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:49.833132 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0eea15-15c2-464e-b4b7-45aead0afab0 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:49.833408 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0eea15-15c2-464e-b4b7-45aead0afab0 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:49.833678 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0eea15-15c2-464e-b4b7-45aead0afab0 None admin] PATH_INFO: `/v3/domains` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:49.834322 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0eea15-15c2-464e-b4b7-45aead0afab0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:49.834622 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0eea15-15c2-464e-b4b7-45aead0afab0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:49.835988 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0eea15-15c2-464e-b4b7-45aead0afab0 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:49.848767 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 289/585] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:50:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:49.850302 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0eea15-15c2-464e-b4b7-45aead0afab0 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:49.876116 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48d92a4-50ad-476f-ada3-11d2e59ca136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:49.876784 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d92a4-50ad-476f-ada3-11d2e59ca136 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:49.877067 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d92a4-50ad-476f-ada3-11d2e59ca136 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:49.877343 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d92a4-50ad-476f-ada3-11d2e59ca136 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:49.878067 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48d92a4-50ad-476f-ada3-11d2e59ca136 None admin] RBAC: Adding query filter params (role.id=d90649d4ef604eeb8edcf0aa22932ca1, scope.project.id=4bb6307eb0644d85a1d1daaa61352697, user.id=e7fc9da7505744608dfd3bbd46a0a9d8) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:49.878374 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48d92a4-50ad-476f-ada3-11d2e59ca136 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:49.879817 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48d92a4-50ad-476f-ada3-11d2e59ca136 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:49.888686 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 297/586] 10.222.0.18 () {56 vars in 1374 bytes} [Sat Aug 6 21:50:49 2022] GET /identity/v3/role_assignments?user.id=e7fc9da7505744608dfd3bbd46a0a9d8&scope.project.id=4bb6307eb0644d85a1d1daaa61352697&role.id=d90649d4ef604eeb8edcf0aa22932ca1 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:49.888686 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48d92a4-50ad-476f-ada3-11d2e59ca136 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:49.971446 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0eea15-15c2-464e-b4b7-45aead0afab0 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:50.013385 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0c343c-a0db-4dc0-8114-5a3e19f42346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:50.013990 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:50.015189 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0c343c-a0db-4dc0-8114-5a3e19f42346 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:50.015643 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0c343c-a0db-4dc0-8114-5a3e19f42346 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:50.016245 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0c343c-a0db-4dc0-8114-5a3e19f42346 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:50.028847 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0c343c-a0db-4dc0-8114-5a3e19f42346 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:50.030709 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0c343c-a0db-4dc0-8114-5a3e19f42346 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:50.033717 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48d92a4-50ad-476f-ada3-11d2e59ca136 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:50.049458 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-7d0c343c-a0db-4dc0-8114-5a3e19f42346 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:50:50.049458 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:50.049458 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:50.049458 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:50.049458 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:50.049458 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:50.049458 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:50.049458 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:50.049458 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:50.049458 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:50.049458 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:50.049458 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:50.049458 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:50:50.049458 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:50:50.049458 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:50:50.049458 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:50:50.050649 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:50.050649 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:50.050649 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:50.050649 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:50.050649 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:50.050649 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:50.050649 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:50.050649 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:50.050649 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:50.050649 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:50.050649 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:50.050649 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:50.050649 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:50.050649 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:50.050649 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:50.050649 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:50.050649 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:50:50.051782 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:50:50.051782 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:50:50.051782 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:50:50.053660 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 290/587] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:50:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:50.054850 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0c343c-a0db-4dc0-8114-5a3e19f42346 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:50.079937 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fb2d42-bf7b-4504-80f7-e825db6d548b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:50.080751 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fb2d42-bf7b-4504-80f7-e825db6d548b None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:50.081102 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fb2d42-bf7b-4504-80f7-e825db6d548b None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:50.081447 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fb2d42-bf7b-4504-80f7-e825db6d548b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:50.096587 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fb2d42-bf7b-4504-80f7-e825db6d548b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:50.097548 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:50.098813 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fb2d42-bf7b-4504-80f7-e825db6d548b None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:50.118497 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-37fb2d42-bf7b-4504-80f7-e825db6d548b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:50:50.118497 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:50.118497 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:50.118497 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:50.118497 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:50.118497 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:50.118497 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:50.118497 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:50.118497 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:50.118497 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:50.118497 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:50.118497 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:50.118497 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:50:50.118497 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:50:50.118497 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:50:50.118497 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:50:50.119503 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:50.119503 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:50.119503 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:50.119503 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:50.119503 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:50.119503 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:50.119503 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:50.119503 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:50.119503 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:50.119503 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:50.119503 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:50.119503 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:50.119503 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:50.119503 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:50.119503 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:50.119503 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:50.119503 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:50:50.120581 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:50:50.120581 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:50:50.120581 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:50:50.120581 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 298/588] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:50:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:50.120808 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fb2d42-bf7b-4504-80f7-e825db6d548b None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:50.199026 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0c343c-a0db-4dc0-8114-5a3e19f42346 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:50.236649 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296b552f-8b9a-4b93-b97c-39e15f45d769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:50.237639 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296b552f-8b9a-4b93-b97c-39e15f45d769 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:50.238162 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296b552f-8b9a-4b93-b97c-39e15f45d769 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:50.238651 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296b552f-8b9a-4b93-b97c-39e15f45d769 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:50.246611 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296b552f-8b9a-4b93-b97c-39e15f45d769 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:50.248660 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296b552f-8b9a-4b93-b97c-39e15f45d769 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:50.254548 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fb2d42-bf7b-4504-80f7-e825db6d548b None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:50.261612 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-296b552f-8b9a-4b93-b97c-39e15f45d769 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 06 21:50:50.261612 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:50.261612 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:50.261612 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:50.261612 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:50.261612 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:50.261612 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:50.261612 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:50.261612 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:50.261612 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:50.261612 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:50.261612 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:50.261612 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 21:50:50.261612 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 21:50:50.261612 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 21:50:50.261612 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 21:50:50.262567 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:50.262567 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:50.262567 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 21:50:50.262567 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 21:50:50.262567 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 21:50:50.262567 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 21:50:50.262567 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:50.262567 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:50.262567 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:50.262567 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:50.262567 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:50.262567 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:50.262567 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:50.262567 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:50.262567 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:50.262567 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:50.262567 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:50.262567 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:50.263643 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:50.263643 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:50.263643 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 21:50:50.263643 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 21:50:50.263643 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 21:50:50.263643 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 21:50:50.263643 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 21:50:50.263643 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 21:50:50.263643 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 06 21:50:50.263643 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:50:50.265840 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 291/589] 10.222.0.18 () {56 vars in 1099 bytes} [Sat Aug 6 21:50:50 2022] GET /identity/v3/users/cinder => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:50.267256 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296b552f-8b9a-4b93-b97c-39e15f45d769 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:50.298687 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a946040-d29f-4097-a2fc-4340c0895bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:50.299298 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a946040-d29f-4097-a2fc-4340c0895bb0 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:50.299630 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a946040-d29f-4097-a2fc-4340c0895bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:50.299892 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a946040-d29f-4097-a2fc-4340c0895bb0 None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:50.300894 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a946040-d29f-4097-a2fc-4340c0895bb0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:50.301204 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a946040-d29f-4097-a2fc-4340c0895bb0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:50.303280 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a946040-d29f-4097-a2fc-4340c0895bb0 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:50.306767 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:50.320863 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 299/590] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:50:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:50.321613 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a946040-d29f-4097-a2fc-4340c0895bb0 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:50.406722 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:50.414933 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296b552f-8b9a-4b93-b97c-39e15f45d769 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:50.450303 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a946040-d29f-4097-a2fc-4340c0895bb0 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:50.455103 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987fe887-c599-4fb3-bc93-75f699c6323f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:50.455968 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987fe887-c599-4fb3-bc93-75f699c6323f None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:50.456507 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987fe887-c599-4fb3-bc93-75f699c6323f None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:50.456989 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987fe887-c599-4fb3-bc93-75f699c6323f None admin] PATH_INFO: `/v3/domains` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:50.457904 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987fe887-c599-4fb3-bc93-75f699c6323f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:50.458398 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987fe887-c599-4fb3-bc93-75f699c6323f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:50.459967 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987fe887-c599-4fb3-bc93-75f699c6323f None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:50.472071 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 292/591] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:50:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:50.473242 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987fe887-c599-4fb3-bc93-75f699c6323f None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:50.494204 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d66642-4b28-4cb6-b62d-0cc02ab45f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:50.494815 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d66642-4b28-4cb6-b62d-0cc02ab45f10 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:50.495095 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d66642-4b28-4cb6-b62d-0cc02ab45f10 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:50.495365 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d66642-4b28-4cb6-b62d-0cc02ab45f10 None admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:50.496092 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d66642-4b28-4cb6-b62d-0cc02ab45f10 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:50.496394 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d66642-4b28-4cb6-b62d-0cc02ab45f10 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:50.497875 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d66642-4b28-4cb6-b62d-0cc02ab45f10 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:50.522507 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 300/592] 10.222.0.18 () {56 vars in 1137 bytes} [Sat Aug 6 21:50:50 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:50.523345 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d66642-4b28-4cb6-b62d-0cc02ab45f10 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:50.585005 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:50.597851 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987fe887-c599-4fb3-bc93-75f699c6323f None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:50.632206 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d66642-4b28-4cb6-b62d-0cc02ab45f10 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:50.635480 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8223117d-8ab0-4390-968e-dc06e3db1e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:50.635796 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223117d-8ab0-4390-968e-dc06e3db1e74 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:50.636088 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223117d-8ab0-4390-968e-dc06e3db1e74 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:50.636350 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223117d-8ab0-4390-968e-dc06e3db1e74 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:50.644045 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8223117d-8ab0-4390-968e-dc06e3db1e74 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:50.645614 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8223117d-8ab0-4390-968e-dc06e3db1e74 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:50.657703 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-8223117d-8ab0-4390-968e-dc06e3db1e74 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 21:50:50.657703 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:50.657703 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:50.657703 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:50.657703 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:50.657703 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:50.657703 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:50.657703 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:50.657703 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:50.657703 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:50.657703 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:50.657703 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:50.657703 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 21:50:50.657703 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 21:50:50.657703 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 21:50:50.659072 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 21:50:50.659072 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:50.659072 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:50.659072 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:50.659072 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:50.659072 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:50.659072 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:50.659072 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:50.659072 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:50.659072 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:50.659072 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:50.659072 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:50.659072 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:50.659072 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:50.659072 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:50.659072 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:50.659072 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:50.660243 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 21:50:50.660243 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 21:50:50.660243 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 21:50:50.660243 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 21:50:50.660243 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 21:50:50.660243 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 21:50:50.660243 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 21:50:50.660243 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:50:50.663559 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 293/593] 10.222.0.18 () {56 vars in 1111 bytes} [Sat Aug 6 21:50:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:50.664554 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8223117d-8ab0-4390-968e-dc06e3db1e74 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:50.676118 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4854d396-497b-4a7c-ad6f-cc5fa4985331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:50.677148 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4854d396-497b-4a7c-ad6f-cc5fa4985331 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:50.677651 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4854d396-497b-4a7c-ad6f-cc5fa4985331 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:50.678164 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4854d396-497b-4a7c-ad6f-cc5fa4985331 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:50.685272 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:50.686759 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4854d396-497b-4a7c-ad6f-cc5fa4985331 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:50.688587 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4854d396-497b-4a7c-ad6f-cc5fa4985331 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:50.700915 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-4854d396-497b-4a7c-ad6f-cc5fa4985331 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 06 21:50:50.700915 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:50.700915 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:50.700915 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:50.700915 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:50.700915 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:50.700915 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:50.700915 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:50.700915 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:50.700915 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:50.700915 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:50.700915 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:50.700915 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 21:50:50.700915 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 21:50:50.700915 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 21:50:50.700915 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 21:50:50.701885 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:50.701885 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:50.701885 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 21:50:50.701885 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 21:50:50.701885 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 21:50:50.701885 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 21:50:50.701885 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:50.701885 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:50.701885 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:50.701885 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:50.701885 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:50.701885 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:50.701885 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:50.701885 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:50.701885 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:50.701885 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:50.701885 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:50.701885 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:50.702809 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:50.702809 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:50.702809 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 21:50:50.702809 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 21:50:50.702809 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 21:50:50.702809 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 21:50:50.702809 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 21:50:50.702809 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 21:50:50.702809 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 06 21:50:50.702809 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:50:50.706147 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 301/594] 10.222.0.18 () {56 vars in 1102 bytes} [Sat Aug 6 21:50:50 2022] GET /identity/v3/users/neutron => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:50.707246 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4854d396-497b-4a7c-ad6f-cc5fa4985331 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:50.774162 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8223117d-8ab0-4390-968e-dc06e3db1e74 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:50.810542 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e81923-65e8-4ef9-9f6b-fcfd6211c2aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:50.811045 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e81923-65e8-4ef9-9f6b-fcfd6211c2aa None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:50.811328 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e81923-65e8-4ef9-9f6b-fcfd6211c2aa None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:50.811601 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e81923-65e8-4ef9-9f6b-fcfd6211c2aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:50.814298 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4854d396-497b-4a7c-ad6f-cc5fa4985331 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:50.822117 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e81923-65e8-4ef9-9f6b-fcfd6211c2aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:50.823561 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e81923-65e8-4ef9-9f6b-fcfd6211c2aa None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:50.830461 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:50.838746 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-95e81923-65e8-4ef9-9f6b-fcfd6211c2aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:50:50.838746 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:50.838746 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:50.838746 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:50.838746 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:50.838746 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:50.838746 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:50.838746 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:50.838746 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:50.838746 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:50.838746 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:50.838746 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:50.838746 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:50:50.838746 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:50:50.838746 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:50:50.838746 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:50:50.839940 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:50.839940 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:50.839940 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:50.839940 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:50.839940 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:50.839940 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:50.839940 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:50.839940 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:50.839940 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:50.839940 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:50.839940 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:50.839940 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:50.839940 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:50.839940 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:50.839940 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:50.839940 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:50.839940 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:50:50.846456 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:50:50.846456 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:50:50.846456 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:50:50.846456 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 294/595] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:50:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:50.846456 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e81923-65e8-4ef9-9f6b-fcfd6211c2aa None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:50.859424 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4750be-07d1-4f23-b1bb-512a9535cf5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:50.860688 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4750be-07d1-4f23-b1bb-512a9535cf5b None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:50.860970 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4750be-07d1-4f23-b1bb-512a9535cf5b None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:50.861241 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4750be-07d1-4f23-b1bb-512a9535cf5b None admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:50.862099 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4750be-07d1-4f23-b1bb-512a9535cf5b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:50.862406 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4750be-07d1-4f23-b1bb-512a9535cf5b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:50.864608 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4750be-07d1-4f23-b1bb-512a9535cf5b None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:50.884473 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 302/596] 10.222.0.18 () {56 vars in 1148 bytes} [Sat Aug 6 21:50:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:50.886160 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4750be-07d1-4f23-b1bb-512a9535cf5b None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:50.932878 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:50.974100 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e81923-65e8-4ef9-9f6b-fcfd6211c2aa None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:51.016577 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4750be-07d1-4f23-b1bb-512a9535cf5b None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:51.016797 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baed5d6e-b310-415f-87ea-373485603271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:51.017125 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baed5d6e-b310-415f-87ea-373485603271 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:51.017407 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baed5d6e-b310-415f-87ea-373485603271 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:51.017688 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baed5d6e-b310-415f-87ea-373485603271 None admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:51.018422 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baed5d6e-b310-415f-87ea-373485603271 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:51.019441 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baed5d6e-b310-415f-87ea-373485603271 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:51.020164 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baed5d6e-b310-415f-87ea-373485603271 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:51.043116 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 295/597] 10.222.0.18 () {56 vars in 1139 bytes} [Sat Aug 6 21:50:50 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:51.044016 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baed5d6e-b310-415f-87ea-373485603271 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:51.061398 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e5c150-c583-4399-9f88-82a47e17000e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:51.062625 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e5c150-c583-4399-9f88-82a47e17000e None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:51.064184 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e5c150-c583-4399-9f88-82a47e17000e None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:51.064712 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e5c150-c583-4399-9f88-82a47e17000e None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:51.065655 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e5c150-c583-4399-9f88-82a47e17000e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:51.066158 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e5c150-c583-4399-9f88-82a47e17000e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:51.068314 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e5c150-c583-4399-9f88-82a47e17000e None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:51.084091 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 303/598] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:50:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:51.085216 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e5c150-c583-4399-9f88-82a47e17000e None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:51.123803 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:51.204274 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e5c150-c583-4399-9f88-82a47e17000e None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:51.245525 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1384d23-c798-48fa-a618-6400567d5043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:51.246715 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1384d23-c798-48fa-a618-6400567d5043 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:51.247489 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1384d23-c798-48fa-a618-6400567d5043 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:51.248230 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1384d23-c798-48fa-a618-6400567d5043 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:51.263371 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1384d23-c798-48fa-a618-6400567d5043 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:51.265149 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1384d23-c798-48fa-a618-6400567d5043 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:51.284198 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-d1384d23-c798-48fa-a618-6400567d5043 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:50:51.284198 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:51.284198 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:51.284198 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:51.284198 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:51.284198 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:51.284198 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:51.284198 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:51.284198 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:51.284198 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:51.284198 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:51.284198 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:51.284198 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:50:51.284198 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:50:51.284198 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:50:51.284198 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:50:51.285200 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:51.285200 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:51.285200 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:51.285200 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:51.285200 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:51.285200 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:51.285200 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:51.285200 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:51.285200 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:51.285200 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:51.285200 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:51.285200 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:51.285200 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:51.285200 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:51.285200 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:51.285200 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:51.285200 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:50:51.286147 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:50:51.286147 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:50:51.286147 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:50:51.292853 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 304/599] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:50:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:51.292853 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1384d23-c798-48fa-a618-6400567d5043 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:51.300185 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baed5d6e-b310-415f-87ea-373485603271 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:51.348780 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:51.350356 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da39364-26f9-4dc3-8294-ac4903f12365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:51.351034 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da39364-26f9-4dc3-8294-ac4903f12365 None admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:51.351308 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da39364-26f9-4dc3-8294-ac4903f12365 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:51.351583 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da39364-26f9-4dc3-8294-ac4903f12365 None admin] PATH_INFO: `/v3/projects/4bb6307eb0644d85a1d1daaa61352697/users/c8d38e73e105476e8de858cfa939ba20/roles/d90649d4ef604eeb8edcf0aa22932ca1` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:51.373150 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da39364-26f9-4dc3-8294-ac4903f12365 None admin] RBAC: Authorizing `identity:create_grant(project_id=4bb6307eb0644d85a1d1daaa61352697, user_id=c8d38e73e105476e8de858cfa939ba20, role_id=d90649d4ef604eeb8edcf0aa22932ca1)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:51.374856 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da39364-26f9-4dc3-8294-ac4903f12365 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:51.397805 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:51.413713 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1384d23-c798-48fa-a618-6400567d5043 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:51.452442 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 296/600] 10.222.0.18 () {58 vars in 1439 bytes} [Sat Aug 6 21:50:51 2022] PUT /identity/v3/projects/4bb6307eb0644d85a1d1daaa61352697/users/c8d38e73e105476e8de858cfa939ba20/roles/d90649d4ef604eeb8edcf0aa22932ca1 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 21:50:51.452442 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da39364-26f9-4dc3-8294-ac4903f12365 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:51.475634 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131ce8b1-ba90-44c5-aad6-7224ba43385a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:51.476322 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131ce8b1-ba90-44c5-aad6-7224ba43385a None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:51.476581 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131ce8b1-ba90-44c5-aad6-7224ba43385a None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:51.476857 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131ce8b1-ba90-44c5-aad6-7224ba43385a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:51.492996 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131ce8b1-ba90-44c5-aad6-7224ba43385a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:51.494499 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131ce8b1-ba90-44c5-aad6-7224ba43385a None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:51.512042 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-131ce8b1-ba90-44c5-aad6-7224ba43385a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 21:50:51.512042 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:51.512042 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:51.512042 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:51.512042 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:51.512042 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:51.512042 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:51.512042 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:51.512042 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:51.512042 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:51.512042 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:51.512042 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:51.512042 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 21:50:51.512042 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 21:50:51.512042 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 21:50:51.513039 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 21:50:51.513039 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:51.513039 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:51.513039 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:51.513039 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:51.513039 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:51.513039 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:51.513039 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:51.513039 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:51.513039 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:51.513039 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:51.513039 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:51.513039 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:51.513039 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:51.513039 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:51.513039 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:51.513039 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:51.513949 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 21:50:51.513949 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 21:50:51.513949 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 21:50:51.513949 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 21:50:51.513949 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 21:50:51.513949 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 21:50:51.513949 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 21:50:51.513949 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:50:51.513949 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 305/601] 10.222.0.18 () {56 vars in 1111 bytes} [Sat Aug 6 21:50:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:51.525491 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131ce8b1-ba90-44c5-aad6-7224ba43385a None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:51.629513 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da39364-26f9-4dc3-8294-ac4903f12365 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:51.681452 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e5dd18-c713-4077-9f36-6f406a8f2324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:51.682067 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e5dd18-c713-4077-9f36-6f406a8f2324 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:51.682340 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e5dd18-c713-4077-9f36-6f406a8f2324 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:51.682604 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e5dd18-c713-4077-9f36-6f406a8f2324 None admin] PATH_INFO: `/v3/domains` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:51.683253 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e5dd18-c713-4077-9f36-6f406a8f2324 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:51.683590 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e5dd18-c713-4077-9f36-6f406a8f2324 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:51.685582 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e5dd18-c713-4077-9f36-6f406a8f2324 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:51.701904 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 297/602] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:50:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:51.707896 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e5dd18-c713-4077-9f36-6f406a8f2324 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:51.718266 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131ce8b1-ba90-44c5-aad6-7224ba43385a None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:51.754598 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:51.772022 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab618ac-09a4-42f7-80b0-ca3222dcb0c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:51.772653 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab618ac-09a4-42f7-80b0-ca3222dcb0c1 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:51.772927 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab618ac-09a4-42f7-80b0-ca3222dcb0c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:51.773200 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab618ac-09a4-42f7-80b0-ca3222dcb0c1 None admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:51.773887 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab618ac-09a4-42f7-80b0-ca3222dcb0c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:51.774189 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab618ac-09a4-42f7-80b0-ca3222dcb0c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:51.775636 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab618ac-09a4-42f7-80b0-ca3222dcb0c1 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:51.788679 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 306/603] 10.222.0.18 () {56 vars in 1148 bytes} [Sat Aug 6 21:50:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:51.789840 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524dcb2c-5a0b-4d04-8759-7b8179491193 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:51.794647 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524dcb2c-5a0b-4d04-8759-7b8179491193 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:51.794647 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524dcb2c-5a0b-4d04-8759-7b8179491193 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:51.795716 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 307/604] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:50:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:50:51.796502 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524dcb2c-5a0b-4d04-8759-7b8179491193 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:51.806101 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:51.857052 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e5dd18-c713-4077-9f36-6f406a8f2324 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:51.893035 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b222e8c-c8f9-4dd6-ba8c-1f0ff812cbfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:51.893580 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b222e8c-c8f9-4dd6-ba8c-1f0ff812cbfa None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:51.893857 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b222e8c-c8f9-4dd6-ba8c-1f0ff812cbfa None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:51.894125 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b222e8c-c8f9-4dd6-ba8c-1f0ff812cbfa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:51.901358 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b222e8c-c8f9-4dd6-ba8c-1f0ff812cbfa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:51.903003 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b222e8c-c8f9-4dd6-ba8c-1f0ff812cbfa None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:51.915012 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-6b222e8c-c8f9-4dd6-ba8c-1f0ff812cbfa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 21:50:51.915012 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:51.915012 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:51.915012 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:51.915012 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:51.915012 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:51.915012 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:51.915012 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:51.915012 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:51.915012 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:51.915012 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:51.915012 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:51.915012 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 21:50:51.915012 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 21:50:51.915012 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 21:50:51.915972 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 21:50:51.915972 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:51.915972 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:51.915972 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:51.915972 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:51.915972 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:51.915972 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:51.915972 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:51.915972 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:51.915972 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:51.915972 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:51.915972 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:51.915972 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:51.915972 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:51.915972 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:51.915972 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:51.915972 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:51.916877 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 21:50:51.916877 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 21:50:51.916877 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 21:50:51.916877 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 21:50:51.916877 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 21:50:51.916877 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 21:50:51.916877 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 21:50:51.916877 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:50:51.916877 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 298/605] 10.222.0.18 () {56 vars in 1111 bytes} [Sat Aug 6 21:50:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:51.917835 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea894b2c-db52-4799-92bd-8b3118a005ef None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:51.918013 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea894b2c-db52-4799-92bd-8b3118a005ef None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:51.918263 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea894b2c-db52-4799-92bd-8b3118a005ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:51.927033 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524dcb2c-5a0b-4d04-8759-7b8179491193 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:51.937181 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-ea894b2c-db52-4799-92bd-8b3118a005ef None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:50:51.939324 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-ea894b2c-db52-4799-92bd-8b3118a005ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:50:51.964527 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b36dd5d-1e79-4201-9def-94ece5881e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:51.965583 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b36dd5d-1e79-4201-9def-94ece5881e2f None admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:51.966125 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b36dd5d-1e79-4201-9def-94ece5881e2f None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:51.966597 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b36dd5d-1e79-4201-9def-94ece5881e2f None admin] PATH_INFO: `/v3/projects/4bb6307eb0644d85a1d1daaa61352697/users/aa71c71de263478da234f3c5716a6e5f/roles/d90649d4ef604eeb8edcf0aa22932ca1` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:51.982504 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b36dd5d-1e79-4201-9def-94ece5881e2f None admin] RBAC: Authorizing `identity:create_grant(project_id=4bb6307eb0644d85a1d1daaa61352697, user_id=aa71c71de263478da234f3c5716a6e5f, role_id=d90649d4ef604eeb8edcf0aa22932ca1)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:51.984303 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b36dd5d-1e79-4201-9def-94ece5881e2f None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:52.013139 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 299/606] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:50:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 21:50:52.017328 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea894b2c-db52-4799-92bd-8b3118a005ef None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:52.050452 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:52.071444 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 308/607] 10.222.0.18 () {58 vars in 1439 bytes} [Sat Aug 6 21:50:51 2022] PUT /identity/v3/projects/4bb6307eb0644d85a1d1daaa61352697/users/aa71c71de263478da234f3c5716a6e5f/roles/d90649d4ef604eeb8edcf0aa22932ca1 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 21:50:52.072308 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b36dd5d-1e79-4201-9def-94ece5881e2f None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:52.150493 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:52.164567 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea894b2c-db52-4799-92bd-8b3118a005ef None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:52.199977 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b36dd5d-1e79-4201-9def-94ece5881e2f None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:52.216087 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9834e3-53e0-4002-84a0-fb3037156311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:52.216764 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9834e3-53e0-4002-84a0-fb3037156311 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:52.217045 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9834e3-53e0-4002-84a0-fb3037156311 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:52.217315 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9834e3-53e0-4002-84a0-fb3037156311 None admin] PATH_INFO: `/v3/projects` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:52.218079 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9834e3-53e0-4002-84a0-fb3037156311 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:52.218384 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9834e3-53e0-4002-84a0-fb3037156311 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:52.219909 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9834e3-53e0-4002-84a0-fb3037156311 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:52.234259 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 300/608] 10.222.0.18 () {56 vars in 1148 bytes} [Sat Aug 6 21:50:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:52.236165 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d141647-4192-4c1c-99dc-fca65faa83e3 None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:52.236473 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d141647-4192-4c1c-99dc-fca65faa83e3 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:52.236749 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d141647-4192-4c1c-99dc-fca65faa83e3 None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:52.238945 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 301/609] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:50:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:50:52.242750 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d141647-4192-4c1c-99dc-fca65faa83e3 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:52.254693 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22b8bc5-ccfe-491c-824a-68cf5223d29f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:52.255826 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22b8bc5-ccfe-491c-824a-68cf5223d29f None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:52.256107 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22b8bc5-ccfe-491c-824a-68cf5223d29f None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:52.256378 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22b8bc5-ccfe-491c-824a-68cf5223d29f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:52.268524 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22b8bc5-ccfe-491c-824a-68cf5223d29f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:50:52.273192 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-f22b8bc5-ccfe-491c-824a-68cf5223d29f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 21:50:52.273192 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:52.273192 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:52.273192 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:52.273192 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:52.273192 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:52.273192 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:52.273192 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:52.273192 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:52.273192 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:52.273192 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:52.273192 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:52.273192 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:50:52.273192 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:50:52.273192 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:50:52.273192 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:50:52.274213 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:50:52.274213 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:50:52.274213 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:50:52.274213 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:50:52.274213 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:52.274213 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:52.274213 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:52.274213 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:52.274213 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:52.274213 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:52.274213 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:52.274213 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:52.274213 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:52.274213 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:52.274213 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:52.274213 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:52.275069 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:52.275069 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:52.275069 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:52.275069 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:52.275069 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:50:52.275069 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:50:52.275069 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:50:52.275069 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:50:52.275069 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:50:52.275069 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:50:52.275069 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 21:50:52.275069 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:50:52.275069 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 309/610] 10.222.0.18 () {56 vars in 1102 bytes} [Sat Aug 6 21:50:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:52.275768 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801a7f9b-01ca-4ab8-99ed-ab201248609b None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:52.275768 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801a7f9b-01ca-4ab8-99ed-ab201248609b None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:52.276771 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801a7f9b-01ca-4ab8-99ed-ab201248609b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:52.295204 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-801a7f9b-01ca-4ab8-99ed-ab201248609b None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:50:52.296485 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-801a7f9b-01ca-4ab8-99ed-ab201248609b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:50:52.363318 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:52.379487 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d141647-4192-4c1c-99dc-fca65faa83e3 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:52.380152 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 310/611] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:50:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 21:50:52.381111 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801a7f9b-01ca-4ab8-99ed-ab201248609b None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:52.416551 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642ca3e9-8fe1-43c0-807d-0ea98ec00993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:52.417326 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642ca3e9-8fe1-43c0-807d-0ea98ec00993 None admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:52.417703 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642ca3e9-8fe1-43c0-807d-0ea98ec00993 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:52.418108 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642ca3e9-8fe1-43c0-807d-0ea98ec00993 None admin] PATH_INFO: `/v3/projects/4bb6307eb0644d85a1d1daaa61352697/users/5ceaeadc843c4186b25114b28607fe22/roles/d90649d4ef604eeb8edcf0aa22932ca1` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:52.434263 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642ca3e9-8fe1-43c0-807d-0ea98ec00993 None admin] RBAC: Authorizing `identity:create_grant(project_id=4bb6307eb0644d85a1d1daaa61352697, user_id=5ceaeadc843c4186b25114b28607fe22, role_id=d90649d4ef604eeb8edcf0aa22932ca1)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:52.436616 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642ca3e9-8fe1-43c0-807d-0ea98ec00993 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:52.440628 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:52.495934 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801a7f9b-01ca-4ab8-99ed-ab201248609b None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:52.528437 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 302/612] 10.222.0.18 () {58 vars in 1439 bytes} [Sat Aug 6 21:50:52 2022] PUT /identity/v3/projects/4bb6307eb0644d85a1d1daaa61352697/users/5ceaeadc843c4186b25114b28607fe22/roles/d90649d4ef604eeb8edcf0aa22932ca1 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 21:50:52.529932 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642ca3e9-8fe1-43c0-807d-0ea98ec00993 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:52.542470 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4c2b38-5f2c-41e4-a574-0dad892d4afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:52.543348 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c2b38-5f2c-41e4-a574-0dad892d4afa None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:52.543768 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c2b38-5f2c-41e4-a574-0dad892d4afa None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:52.544152 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c2b38-5f2c-41e4-a574-0dad892d4afa None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:52.545003 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4c2b38-5f2c-41e4-a574-0dad892d4afa None admin] RBAC: Adding query filter params (name=service) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:52.545403 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4c2b38-5f2c-41e4-a574-0dad892d4afa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:52.547008 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4c2b38-5f2c-41e4-a574-0dad892d4afa None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:52.575572 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 311/613] 10.222.0.18 () {56 vars in 1103 bytes} [Sat Aug 6 21:50:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:52.577100 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4c2b38-5f2c-41e4-a574-0dad892d4afa None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:52.719230 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642ca3e9-8fe1-43c0-807d-0ea98ec00993 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:52.773015 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3aaa88-ab23-4ff4-a5b2-f1cc04497411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:52.773730 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3aaa88-ab23-4ff4-a5b2-f1cc04497411 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:52.774005 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3aaa88-ab23-4ff4-a5b2-f1cc04497411 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:52.774274 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3aaa88-ab23-4ff4-a5b2-f1cc04497411 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:52.776006 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4c2b38-5f2c-41e4-a574-0dad892d4afa None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:52.779449 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:52.786966 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:52.788706 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3aaa88-ab23-4ff4-a5b2-f1cc04497411 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:50:52.793610 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-3e3aaa88-ab23-4ff4-a5b2-f1cc04497411 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 21:50:52.793610 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:52.793610 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:52.793610 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:52.793610 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:52.793610 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:52.793610 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:52.793610 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:52.793610 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:52.793610 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:52.793610 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:52.793610 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:52.793610 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:50:52.793610 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:50:52.793610 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:50:52.793610 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:50:52.797922 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:50:52.797922 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:50:52.797922 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:50:52.797922 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:50:52.797922 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:52.797922 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:52.797922 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:52.797922 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:52.797922 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:52.797922 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:52.797922 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:52.797922 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:52.797922 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:52.797922 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:52.797922 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:52.797922 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:52.798759 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:52.798759 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:52.798759 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:52.798759 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:52.798759 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:50:52.798759 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:50:52.798759 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:50:52.798759 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:50:52.798759 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:50:52.798759 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:50:52.798759 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 21:50:52.798759 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:50:52.798759 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 303/614] 10.222.0.18 () {56 vars in 1102 bytes} [Sat Aug 6 21:50:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:52.802334 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3aaa88-ab23-4ff4-a5b2-f1cc04497411 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:52.817418 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138bdd4c-37a9-4a7e-9fc0-5147b456b646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:52.819624 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138bdd4c-37a9-4a7e-9fc0-5147b456b646 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:52.819624 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138bdd4c-37a9-4a7e-9fc0-5147b456b646 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:52.819624 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138bdd4c-37a9-4a7e-9fc0-5147b456b646 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:52.830350 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138bdd4c-37a9-4a7e-9fc0-5147b456b646 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:52.831825 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138bdd4c-37a9-4a7e-9fc0-5147b456b646 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:52.846917 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-138bdd4c-37a9-4a7e-9fc0-5147b456b646 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:50:52.846917 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:52.846917 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:52.846917 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:52.846917 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:52.846917 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:52.846917 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:52.846917 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:52.846917 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:52.846917 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:52.846917 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:52.846917 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:52.846917 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:50:52.846917 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:50:52.846917 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:50:52.846917 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:50:52.847900 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:52.847900 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:52.847900 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:52.847900 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:52.847900 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:52.847900 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:52.847900 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:52.847900 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:52.847900 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:52.847900 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:52.847900 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:52.847900 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:52.847900 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:52.847900 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:52.847900 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:52.847900 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:52.847900 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:50:52.848819 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:50:52.848819 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:50:52.848819 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:50:52.852157 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 312/615] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:50:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:52.854030 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138bdd4c-37a9-4a7e-9fc0-5147b456b646 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:52.944833 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3aaa88-ab23-4ff4-a5b2-f1cc04497411 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:52.981089 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2527f6-526b-4fd6-b203-d58ba192474c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:52.981932 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2527f6-526b-4fd6-b203-d58ba192474c None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:52.982612 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2527f6-526b-4fd6-b203-d58ba192474c None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:52.983104 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2527f6-526b-4fd6-b203-d58ba192474c None admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:52.983979 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2527f6-526b-4fd6-b203-d58ba192474c None admin] RBAC: Adding query filter params (name=service) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:52.984447 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2527f6-526b-4fd6-b203-d58ba192474c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:52.986936 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2527f6-526b-4fd6-b203-d58ba192474c None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:52.995659 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 304/616] 10.222.0.18 () {56 vars in 1103 bytes} [Sat Aug 6 21:50:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:50:53.009007 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2527f6-526b-4fd6-b203-d58ba192474c None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:53.059755 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138bdd4c-37a9-4a7e-9fc0-5147b456b646 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:53.112808 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:53.122308 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e38fd43-2b7e-4e3e-a2fc-9addacaf4d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:53.122918 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e38fd43-2b7e-4e3e-a2fc-9addacaf4d86 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:53.123200 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e38fd43-2b7e-4e3e-a2fc-9addacaf4d86 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:53.123471 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e38fd43-2b7e-4e3e-a2fc-9addacaf4d86 None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:53.124120 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e38fd43-2b7e-4e3e-a2fc-9addacaf4d86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:53.124450 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e38fd43-2b7e-4e3e-a2fc-9addacaf4d86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:53.126742 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e38fd43-2b7e-4e3e-a2fc-9addacaf4d86 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:53.140395 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 313/617] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:50:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:50:53.147451 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e38fd43-2b7e-4e3e-a2fc-9addacaf4d86 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:53.182870 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2527f6-526b-4fd6-b203-d58ba192474c None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:53.198840 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:53.241031 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191994c2-c682-4849-a0b7-6fc9af4ab152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:53.242026 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191994c2-c682-4849-a0b7-6fc9af4ab152 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:53.242521 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191994c2-c682-4849-a0b7-6fc9af4ab152 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:53.243077 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191994c2-c682-4849-a0b7-6fc9af4ab152 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:53.262630 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191994c2-c682-4849-a0b7-6fc9af4ab152 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:53.266982 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191994c2-c682-4849-a0b7-6fc9af4ab152 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:53.292789 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-191994c2-c682-4849-a0b7-6fc9af4ab152 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:50:53.292789 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:53.292789 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:53.292789 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:53.292789 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:53.292789 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:53.292789 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:53.292789 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:53.292789 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:53.292789 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:53.292789 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:53.292789 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:53.292789 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:50:53.292789 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:50:53.292789 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:50:53.292789 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:50:53.293781 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:53.293781 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:53.293781 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:53.293781 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:53.293781 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:53.293781 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:53.293781 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:53.293781 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:53.293781 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:53.293781 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:53.293781 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:53.293781 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:53.293781 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:53.293781 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:53.293781 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:53.293781 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:53.293781 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:50:53.294658 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:50:53.294658 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:50:53.294658 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:50:53.306026 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 305/618] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:50:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:53.308630 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191994c2-c682-4849-a0b7-6fc9af4ab152 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:53.341858 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e38fd43-2b7e-4e3e-a2fc-9addacaf4d86 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:53.388174 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a4cc8f-bd77-4872-9925-d97cad5f01c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:53.389723 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a4cc8f-bd77-4872-9925-d97cad5f01c1 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:53.390208 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a4cc8f-bd77-4872-9925-d97cad5f01c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:53.390696 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a4cc8f-bd77-4872-9925-d97cad5f01c1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:53.400175 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a4cc8f-bd77-4872-9925-d97cad5f01c1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:53.402612 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a4cc8f-bd77-4872-9925-d97cad5f01c1 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:53.417109 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-04a4cc8f-bd77-4872-9925-d97cad5f01c1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 06 21:50:53.417109 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:53.417109 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:53.417109 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:53.417109 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:53.417109 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:53.417109 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:53.417109 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:53.417109 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:53.417109 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:53.417109 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:53.417109 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:53.417109 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 21:50:53.417109 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 21:50:53.417109 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 21:50:53.417109 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 21:50:53.418122 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:53.418122 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:53.418122 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 21:50:53.418122 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 21:50:53.418122 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 21:50:53.418122 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 21:50:53.418122 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:53.418122 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:53.418122 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:53.418122 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:53.418122 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:53.418122 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:53.418122 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:53.418122 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:53.418122 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:53.418122 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:53.418122 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:53.418122 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:53.419050 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:53.419050 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:53.419050 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 21:50:53.419050 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 21:50:53.419050 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 21:50:53.419050 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 21:50:53.419050 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 21:50:53.419050 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 21:50:53.419050 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 06 21:50:53.419050 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:50:53.422089 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 314/619] 10.222.0.18 () {56 vars in 1093 bytes} [Sat Aug 6 21:50:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 21:50:53.428062 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a4cc8f-bd77-4872-9925-d97cad5f01c1 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:53.520858 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:53.540046 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a4cc8f-bd77-4872-9925-d97cad5f01c1 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:53.572635 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191994c2-c682-4849-a0b7-6fc9af4ab152 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:53.593186 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:53.601359 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1eb993-4956-4f5e-9366-48aa21fef0c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:53.601965 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1eb993-4956-4f5e-9366-48aa21fef0c7 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:53.602235 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1eb993-4956-4f5e-9366-48aa21fef0c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:53.602501 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1eb993-4956-4f5e-9366-48aa21fef0c7 None admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:53.603244 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1eb993-4956-4f5e-9366-48aa21fef0c7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:53.603613 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1eb993-4956-4f5e-9366-48aa21fef0c7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:53.605985 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1eb993-4956-4f5e-9366-48aa21fef0c7 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:53.619179 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ccb5b9-b7fd-4797-8658-49ed7af009c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:53.619783 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ccb5b9-b7fd-4797-8658-49ed7af009c8 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:53.620059 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ccb5b9-b7fd-4797-8658-49ed7af009c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:53.620326 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ccb5b9-b7fd-4797-8658-49ed7af009c8 None admin] PATH_INFO: `/v3/domains` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:53.621048 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ccb5b9-b7fd-4797-8658-49ed7af009c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:53.621347 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ccb5b9-b7fd-4797-8658-49ed7af009c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:53.622753 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ccb5b9-b7fd-4797-8658-49ed7af009c8 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:53.643383 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 315/620] 10.222.0.18 () {56 vars in 1133 bytes} [Sat Aug 6 21:50:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:50:53.645652 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 306/621] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:50:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:53.649004 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1eb993-4956-4f5e-9366-48aa21fef0c7 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:53.652004 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ccb5b9-b7fd-4797-8658-49ed7af009c8 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:53.825150 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1eb993-4956-4f5e-9366-48aa21fef0c7 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:53.839045 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ccb5b9-b7fd-4797-8658-49ed7af009c8 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:53.868695 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23db23c5-2325-43b8-8c75-a579fc0c1c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:53.869300 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23db23c5-2325-43b8-8c75-a579fc0c1c3e None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:53.869577 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23db23c5-2325-43b8-8c75-a579fc0c1c3e None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:53.869894 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23db23c5-2325-43b8-8c75-a579fc0c1c3e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:53.880197 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:53.919603 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375b83af-5e70-4706-83bf-f58b123d5441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:53.920268 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375b83af-5e70-4706-83bf-f58b123d5441 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:53.920581 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375b83af-5e70-4706-83bf-f58b123d5441 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:53.920855 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375b83af-5e70-4706-83bf-f58b123d5441 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:53.928829 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375b83af-5e70-4706-83bf-f58b123d5441 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:53.930388 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375b83af-5e70-4706-83bf-f58b123d5441 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:53.943399 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-375b83af-5e70-4706-83bf-f58b123d5441 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 06 21:50:53.943399 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:53.943399 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:53.943399 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:53.943399 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:53.943399 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:53.943399 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:53.943399 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:53.943399 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:53.943399 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:53.943399 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:53.943399 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:53.943399 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 21:50:53.943399 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 21:50:53.943399 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 21:50:53.943399 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 21:50:53.944348 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:53.944348 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:53.944348 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 21:50:53.944348 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 21:50:53.944348 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 21:50:53.944348 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 21:50:53.944348 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:53.944348 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:53.944348 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:53.944348 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:53.944348 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:53.944348 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:53.944348 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:53.944348 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:53.944348 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:53.944348 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:53.944348 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:53.944348 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:53.945319 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:53.945319 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:53.945319 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 21:50:53.945319 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 21:50:53.945319 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 21:50:53.945319 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 21:50:53.945319 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 21:50:53.945319 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 21:50:53.945319 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 06 21:50:53.945319 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:50:53.945319 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 307/622] 10.222.0.18 () {56 vars in 1096 bytes} [Sat Aug 6 21:50:53 2022] GET /identity/v3/users/swift => generated 83 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:53.946404 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637d2fc8-3947-47eb-9bf5-073da7e345bc None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:53.946560 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637d2fc8-3947-47eb-9bf5-073da7e345bc None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:53.951001 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637d2fc8-3947-47eb-9bf5-073da7e345bc None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:53.951513 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23db23c5-2325-43b8-8c75-a579fc0c1c3e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:53.952089 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 308/623] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:50:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:50:53.952914 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637d2fc8-3947-47eb-9bf5-073da7e345bc None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:53.953096 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23db23c5-2325-43b8-8c75-a579fc0c1c3e None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:53.991764 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:54.080289 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 316/624] 10.222.0.18 () {58 vars in 1282 bytes} [Sat Aug 6 21:50:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 21:50:54.081593 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4776c3b-686e-4a30-9e8a-38aa083c6ae1 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:54.081884 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4776c3b-686e-4a30-9e8a-38aa083c6ae1 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:54.082162 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4776c3b-686e-4a30-9e8a-38aa083c6ae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:54.089078 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637d2fc8-3947-47eb-9bf5-073da7e345bc None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:54.103467 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-f4776c3b-686e-4a30-9e8a-38aa083c6ae1 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:50:54.104702 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-f4776c3b-686e-4a30-9e8a-38aa083c6ae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:50:54.125632 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caab51a0-fe5f-45d9-a235-712d88ea7e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:54.126423 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caab51a0-fe5f-45d9-a235-712d88ea7e55 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:54.126698 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caab51a0-fe5f-45d9-a235-712d88ea7e55 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:54.126969 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caab51a0-fe5f-45d9-a235-712d88ea7e55 None admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:54.127695 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caab51a0-fe5f-45d9-a235-712d88ea7e55 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:54.127995 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caab51a0-fe5f-45d9-a235-712d88ea7e55 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:54.129440 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caab51a0-fe5f-45d9-a235-712d88ea7e55 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:54.138088 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:54.149064 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 309/625] 10.222.0.18 () {56 vars in 1135 bytes} [Sat Aug 6 21:50:53 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:54.150633 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caab51a0-fe5f-45d9-a235-712d88ea7e55 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:54.180503 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 317/626] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:50:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 21:50:54.181384 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4776c3b-686e-4a30-9e8a-38aa083c6ae1 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:54.261972 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:54.279424 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caab51a0-fe5f-45d9-a235-712d88ea7e55 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:54.317171 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4776c3b-686e-4a30-9e8a-38aa083c6ae1 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:54.327585 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095a2e03-42b6-4ea6-a82c-d22a146117df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:54.328821 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095a2e03-42b6-4ea6-a82c-d22a146117df None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:54.329618 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095a2e03-42b6-4ea6-a82c-d22a146117df None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:54.329908 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095a2e03-42b6-4ea6-a82c-d22a146117df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:54.344344 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095a2e03-42b6-4ea6-a82c-d22a146117df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:54.348703 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095a2e03-42b6-4ea6-a82c-d22a146117df None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:54.358854 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8860785-3fa3-4f2e-8c0c-868615615d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:54.360011 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8860785-3fa3-4f2e-8c0c-868615615d66 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:54.360288 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8860785-3fa3-4f2e-8c0c-868615615d66 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:54.360612 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8860785-3fa3-4f2e-8c0c-868615615d66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:54.372782 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-095a2e03-42b6-4ea6-a82c-d22a146117df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:50:54.372782 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:54.372782 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:54.372782 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:54.372782 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:54.372782 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:54.372782 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:54.372782 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:54.372782 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:54.372782 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:54.372782 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:54.372782 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:54.372782 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:50:54.372782 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:50:54.372782 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:50:54.372782 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:50:54.373763 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:54.373763 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:54.373763 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:54.373763 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:54.373763 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:54.373763 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:54.373763 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:54.373763 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:54.373763 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:54.373763 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:54.373763 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:54.373763 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:54.373763 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:54.373763 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:54.373763 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:54.373763 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:54.373763 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:50:54.374642 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:50:54.374642 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:50:54.374642 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:50:54.374823 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8860785-3fa3-4f2e-8c0c-868615615d66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:54.380426 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8860785-3fa3-4f2e-8c0c-868615615d66 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:54.380643 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 310/627] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:50:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:54.380643 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095a2e03-42b6-4ea6-a82c-d22a146117df None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:54.394094 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-c8860785-3fa3-4f2e-8c0c-868615615d66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:50:54.394094 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:54.394094 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:54.394094 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:54.394094 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:54.394094 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:54.394094 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:54.394094 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:54.394094 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:54.394094 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:54.394094 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:54.394094 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:54.394094 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:50:54.394094 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:50:54.394094 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:50:54.394094 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:50:54.396245 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:54.396245 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:54.396245 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:54.396245 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:54.396245 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:54.396245 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:54.396245 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:54.396245 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:54.396245 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:54.396245 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:54.396245 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:54.396245 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:54.396245 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:54.396245 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:54.396245 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:54.396245 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:54.396245 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:50:54.398454 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:50:54.398454 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:50:54.398454 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:50:54.404438 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 318/628] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:50:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:54.404438 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8860785-3fa3-4f2e-8c0c-868615615d66 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:54.430320 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:54.503494 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095a2e03-42b6-4ea6-a82c-d22a146117df None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:54.528956 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8860785-3fa3-4f2e-8c0c-868615615d66 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:54.538872 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9070ce74-1da5-488a-9a18-fc7c94cc59c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:54.539497 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9070ce74-1da5-488a-9a18-fc7c94cc59c2 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:54.539778 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9070ce74-1da5-488a-9a18-fc7c94cc59c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:54.540047 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9070ce74-1da5-488a-9a18-fc7c94cc59c2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:54.549451 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:54.552557 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9070ce74-1da5-488a-9a18-fc7c94cc59c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:50:54.558021 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-9070ce74-1da5-488a-9a18-fc7c94cc59c2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 21:50:54.558021 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:54.558021 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:54.558021 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:54.558021 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:54.558021 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:54.558021 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:54.558021 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:54.558021 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:54.558021 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:54.558021 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:54.558021 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:54.558021 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:50:54.558021 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:50:54.558021 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:50:54.558021 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:50:54.558959 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:50:54.558959 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:50:54.558959 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:50:54.558959 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:50:54.558959 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:54.558959 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:54.558959 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:54.558959 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:54.558959 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:54.558959 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:54.558959 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:54.558959 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:54.558959 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:54.558959 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:54.558959 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:54.558959 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:54.559795 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:54.559795 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:54.559795 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:54.559795 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:54.559795 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:50:54.559795 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:50:54.559795 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:50:54.559795 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:50:54.559795 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:50:54.559795 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:50:54.559795 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 21:50:54.559795 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:50:54.559795 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 311/629] 10.222.0.18 () {56 vars in 1102 bytes} [Sat Aug 6 21:50:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:54.560532 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9070ce74-1da5-488a-9a18-fc7c94cc59c2 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:54.572277 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f562d6-33bd-4f03-936f-16bbf16413e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:54.572929 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f562d6-33bd-4f03-936f-16bbf16413e0 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:54.573216 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f562d6-33bd-4f03-936f-16bbf16413e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:54.573491 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f562d6-33bd-4f03-936f-16bbf16413e0 None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:54.574135 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f562d6-33bd-4f03-936f-16bbf16413e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:54.574441 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f562d6-33bd-4f03-936f-16bbf16413e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:54.575856 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f562d6-33bd-4f03-936f-16bbf16413e0 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:54.588637 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 319/630] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:50:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:54.594758 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4314490-4514-4c66-b333-e3da2bbe58e3 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:54.595039 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4314490-4514-4c66-b333-e3da2bbe58e3 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:54.595257 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4314490-4514-4c66-b333-e3da2bbe58e3 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:54.596817 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 320/631] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:50:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:50:54.597533 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4314490-4514-4c66-b333-e3da2bbe58e3 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:54.674615 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9070ce74-1da5-488a-9a18-fc7c94cc59c2 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:54.690851 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:54.710904 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4314490-4514-4c66-b333-e3da2bbe58e3 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:54.711273 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a83885-60c7-4eae-856b-7e79499ecf26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:54.711878 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a83885-60c7-4eae-856b-7e79499ecf26 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:54.712151 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a83885-60c7-4eae-856b-7e79499ecf26 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:54.712432 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a83885-60c7-4eae-856b-7e79499ecf26 None admin] PATH_INFO: `/v3/domains` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:54.713084 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a83885-60c7-4eae-856b-7e79499ecf26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:54.713379 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a83885-60c7-4eae-856b-7e79499ecf26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:54.714736 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a83885-60c7-4eae-856b-7e79499ecf26 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:54.727784 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 312/632] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:50:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:54.729504 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a83885-60c7-4eae-856b-7e79499ecf26 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:54.747659 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fe2875-2549-4858-a777-8b3ceb9cd030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:54.748269 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fe2875-2549-4858-a777-8b3ceb9cd030 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:54.748573 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fe2875-2549-4858-a777-8b3ceb9cd030 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:54.748846 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fe2875-2549-4858-a777-8b3ceb9cd030 None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:54.749507 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fe2875-2549-4858-a777-8b3ceb9cd030 None admin] RBAC: Adding query filter params (name=service) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:54.749865 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fe2875-2549-4858-a777-8b3ceb9cd030 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:54.751756 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fe2875-2549-4858-a777-8b3ceb9cd030 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:54.759780 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 321/633] 10.222.0.18 () {56 vars in 1103 bytes} [Sat Aug 6 21:50:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:54.760934 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3ef8ec-3d9b-4b61-ad17-8a2a3c35d596 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:54.761217 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3ef8ec-3d9b-4b61-ad17-8a2a3c35d596 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:54.761490 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3ef8ec-3d9b-4b61-ad17-8a2a3c35d596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:54.778627 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-aa3ef8ec-3d9b-4b61-ad17-8a2a3c35d596 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:50:54.779826 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-aa3ef8ec-3d9b-4b61-ad17-8a2a3c35d596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:50:54.791315 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:54.832492 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a83885-60c7-4eae-856b-7e79499ecf26 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:54.846608 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 322/634] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:50:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 21:50:54.847372 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3ef8ec-3d9b-4b61-ad17-8a2a3c35d596 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:54.868827 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2e90e7-559f-41bd-b048-c8a4b1b4ab05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:54.869544 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e90e7-559f-41bd-b048-c8a4b1b4ab05 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:54.869776 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e90e7-559f-41bd-b048-c8a4b1b4ab05 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:54.870052 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e90e7-559f-41bd-b048-c8a4b1b4ab05 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:54.878048 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2e90e7-559f-41bd-b048-c8a4b1b4ab05 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:54.879913 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2e90e7-559f-41bd-b048-c8a4b1b4ab05 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:54.892374 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-7d2e90e7-559f-41bd-b048-c8a4b1b4ab05 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 21:50:54.892374 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:54.892374 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:54.892374 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:54.892374 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:54.892374 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:54.892374 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:54.892374 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:54.892374 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:54.892374 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:54.892374 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:54.892374 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:54.892374 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 21:50:54.892374 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 21:50:54.892374 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 21:50:54.893304 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 21:50:54.893304 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:54.893304 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:54.893304 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:54.893304 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:54.893304 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:54.893304 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:54.893304 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:54.893304 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:54.893304 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:54.893304 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:54.893304 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:54.893304 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:54.893304 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:54.893304 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:54.893304 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:54.893304 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:54.894505 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 21:50:54.894505 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 21:50:54.894505 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 21:50:54.894505 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 21:50:54.894505 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 21:50:54.894505 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 21:50:54.894505 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 21:50:54.894505 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:50:54.894505 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 313/635] 10.222.0.18 () {56 vars in 1111 bytes} [Sat Aug 6 21:50:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:54.895320 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221e245e-fcf7-4db9-b96b-82b53652ccf2 None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:54.895320 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221e245e-fcf7-4db9-b96b-82b53652ccf2 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:54.895320 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221e245e-fcf7-4db9-b96b-82b53652ccf2 None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:54.898282 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 314/636] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:50:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:50:54.898282 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221e245e-fcf7-4db9-b96b-82b53652ccf2 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:54.923939 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:54.953871 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3ef8ec-3d9b-4b61-ad17-8a2a3c35d596 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:54.987868 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6e5639-3688-4667-9b05-8828636e6ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:54.988542 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6e5639-3688-4667-9b05-8828636e6ebb None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:54.988823 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6e5639-3688-4667-9b05-8828636e6ebb None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:54.989097 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6e5639-3688-4667-9b05-8828636e6ebb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:54.995531 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6e5639-3688-4667-9b05-8828636e6ebb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:54.997010 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6e5639-3688-4667-9b05-8828636e6ebb None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:54.998905 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221e245e-fcf7-4db9-b96b-82b53652ccf2 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:55.008098 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-7b6e5639-3688-4667-9b05-8828636e6ebb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 21:50:55.008098 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:55.008098 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:55.008098 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:55.008098 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:55.008098 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:55.008098 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:55.008098 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:55.008098 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:55.008098 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:55.008098 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:55.008098 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:55.008098 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 21:50:55.008098 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 21:50:55.008098 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 21:50:55.009090 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 21:50:55.009090 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:55.009090 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:55.009090 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:55.009090 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:55.009090 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:55.009090 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:55.009090 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:55.009090 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:55.009090 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:55.009090 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:55.009090 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:55.009090 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:55.009090 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:55.009090 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:55.009090 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:55.009090 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:55.010017 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 21:50:55.010017 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 21:50:55.010017 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 21:50:55.010017 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 21:50:55.010017 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 21:50:55.010017 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 21:50:55.010017 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 21:50:55.010017 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:50:55.010017 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 323/637] 10.222.0.18 () {56 vars in 1111 bytes} [Sat Aug 6 21:50:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:55.010499 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6e5639-3688-4667-9b05-8828636e6ebb None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:55.023582 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:55.037972 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350ebafc-2b5c-46a4-a525-2d8c5021489b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:55.038897 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350ebafc-2b5c-46a4-a525-2d8c5021489b None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:55.039179 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350ebafc-2b5c-46a4-a525-2d8c5021489b None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:55.039450 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350ebafc-2b5c-46a4-a525-2d8c5021489b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:55.050356 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350ebafc-2b5c-46a4-a525-2d8c5021489b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:55.051878 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350ebafc-2b5c-46a4-a525-2d8c5021489b None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:55.066450 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-350ebafc-2b5c-46a4-a525-2d8c5021489b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:50:55.066450 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:55.066450 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:55.066450 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:55.066450 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:55.066450 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:55.066450 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:55.066450 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:55.066450 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:55.066450 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:55.066450 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:55.066450 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:55.066450 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:50:55.066450 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:50:55.066450 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:50:55.066450 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:50:55.067406 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:55.067406 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:55.067406 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:55.067406 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:55.067406 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:55.067406 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:55.067406 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:55.067406 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:55.067406 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:55.067406 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:55.067406 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:55.067406 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:55.067406 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:55.067406 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:55.067406 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:55.067406 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:55.067406 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:50:55.068351 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:50:55.068351 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:50:55.068351 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:50:55.068351 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 315/638] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:50:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:55.068351 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350ebafc-2b5c-46a4-a525-2d8c5021489b None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:55.113968 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6e5639-3688-4667-9b05-8828636e6ebb None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:55.143845 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:55.148961 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef97d123-c79a-4fbf-af61-d80821d2c1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:55.151852 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef97d123-c79a-4fbf-af61-d80821d2c1ae None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:55.151852 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef97d123-c79a-4fbf-af61-d80821d2c1ae None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:55.151852 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef97d123-c79a-4fbf-af61-d80821d2c1ae None admin] PATH_INFO: `/v3/roles/service` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:55.160299 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef97d123-c79a-4fbf-af61-d80821d2c1ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:50:55.165557 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-ef97d123-c79a-4fbf-af61-d80821d2c1ae None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 21:50:55.165557 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:55.165557 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:55.165557 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:55.165557 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:55.165557 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:55.165557 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:55.165557 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:55.165557 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:55.165557 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:55.165557 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:55.165557 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:55.165557 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:50:55.165557 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:50:55.165557 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:50:55.165557 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:50:55.166547 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:50:55.166547 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:50:55.166547 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:50:55.166547 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:50:55.166547 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:55.166547 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:55.166547 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:55.166547 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:55.166547 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:55.166547 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:55.166547 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:55.166547 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:55.166547 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:55.166547 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:55.166547 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:55.166547 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:55.167605 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:55.167605 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:55.167605 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:55.167605 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:55.167605 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:50:55.167605 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:50:55.167605 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:50:55.167605 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:50:55.167605 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:50:55.167605 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:50:55.167605 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 21:50:55.167605 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:50:55.167605 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 324/639] 10.222.0.18 () {56 vars in 1102 bytes} [Sat Aug 6 21:50:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:55.168449 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e11728-753f-4dc5-a2a6-d6d689e24dec None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:55.168449 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e11728-753f-4dc5-a2a6-d6d689e24dec None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:55.168577 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e11728-753f-4dc5-a2a6-d6d689e24dec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:55.174889 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350ebafc-2b5c-46a4-a525-2d8c5021489b None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:55.187039 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-49e11728-753f-4dc5-a2a6-d6d689e24dec None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:50:55.188837 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-49e11728-753f-4dc5-a2a6-d6d689e24dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:50:55.208883 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349562df-9886-434b-b445-7dae8074d34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:55.209483 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349562df-9886-434b-b445-7dae8074d34d None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:55.209757 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349562df-9886-434b-b445-7dae8074d34d None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:55.210025 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349562df-9886-434b-b445-7dae8074d34d None admin] PATH_INFO: `/v3/projects` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:55.210704 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349562df-9886-434b-b445-7dae8074d34d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:55.211004 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349562df-9886-434b-b445-7dae8074d34d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:55.212527 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349562df-9886-434b-b445-7dae8074d34d None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:55.225090 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 316/640] 10.222.0.18 () {56 vars in 1148 bytes} [Sat Aug 6 21:50:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:55.225893 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349562df-9886-434b-b445-7dae8074d34d None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:55.258358 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 325/641] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:50:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 21:50:55.259202 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e11728-753f-4dc5-a2a6-d6d689e24dec None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:55.282605 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:55.349248 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349562df-9886-434b-b445-7dae8074d34d None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:55.360916 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e11728-753f-4dc5-a2a6-d6d689e24dec None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:55.383942 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:55.384814 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fe182e-c2dc-4814-97ad-1e77db95ee28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:55.385418 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fe182e-c2dc-4814-97ad-1e77db95ee28 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:55.385698 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fe182e-c2dc-4814-97ad-1e77db95ee28 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:55.385966 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fe182e-c2dc-4814-97ad-1e77db95ee28 None admin] PATH_INFO: `/v3/projects` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:55.386646 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fe182e-c2dc-4814-97ad-1e77db95ee28 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:55.386947 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fe182e-c2dc-4814-97ad-1e77db95ee28 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:55.388358 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fe182e-c2dc-4814-97ad-1e77db95ee28 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:55.395174 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764ce780-38e6-4737-b28d-a17cfcac4b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:55.395793 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764ce780-38e6-4737-b28d-a17cfcac4b7e None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:55.396071 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764ce780-38e6-4737-b28d-a17cfcac4b7e None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:55.396366 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764ce780-38e6-4737-b28d-a17cfcac4b7e None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:55.397032 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764ce780-38e6-4737-b28d-a17cfcac4b7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:55.397330 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764ce780-38e6-4737-b28d-a17cfcac4b7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:55.398675 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764ce780-38e6-4737-b28d-a17cfcac4b7e None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:55.399591 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 317/642] 10.222.0.18 () {56 vars in 1148 bytes} [Sat Aug 6 21:50:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:55.400373 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fe182e-c2dc-4814-97ad-1e77db95ee28 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:55.410904 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 326/643] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:50:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:55.411779 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764ce780-38e6-4737-b28d-a17cfcac4b7e None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:55.499525 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fe182e-c2dc-4814-97ad-1e77db95ee28 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:55.506254 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764ce780-38e6-4737-b28d-a17cfcac4b7e None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:55.532851 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771001c3-4b16-4da2-badb-ac07d3aa022b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:55.533450 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771001c3-4b16-4da2-badb-ac07d3aa022b None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:55.533727 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771001c3-4b16-4da2-badb-ac07d3aa022b None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:55.533994 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771001c3-4b16-4da2-badb-ac07d3aa022b None admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:55.534649 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771001c3-4b16-4da2-badb-ac07d3aa022b None admin] RBAC: Adding query filter params (name=service) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:55.534946 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771001c3-4b16-4da2-badb-ac07d3aa022b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:55.536304 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771001c3-4b16-4da2-badb-ac07d3aa022b None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:55.538836 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb6360a-fec0-4641-8c07-978b19586bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:55.539482 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb6360a-fec0-4641-8c07-978b19586bc2 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:55.539708 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb6360a-fec0-4641-8c07-978b19586bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:55.539978 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb6360a-fec0-4641-8c07-978b19586bc2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:55.540721 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb6360a-fec0-4641-8c07-978b19586bc2 None admin] RBAC: Adding query filter params (role.id=d90649d4ef604eeb8edcf0aa22932ca1, scope.project.id=4bb6307eb0644d85a1d1daaa61352697, user.id=2a1319d33e9144859c6010843be312a3) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:55.541033 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb6360a-fec0-4641-8c07-978b19586bc2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:55.542431 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb6360a-fec0-4641-8c07-978b19586bc2 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:55.543914 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 318/644] 10.222.0.18 () {56 vars in 1103 bytes} [Sat Aug 6 21:50:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:55.544725 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771001c3-4b16-4da2-badb-ac07d3aa022b None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:55.548274 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 327/645] 10.222.0.18 () {56 vars in 1374 bytes} [Sat Aug 6 21:50:55 2022] GET /identity/v3/role_assignments?user.id=2a1319d33e9144859c6010843be312a3&scope.project.id=4bb6307eb0644d85a1d1daaa61352697&role.id=d90649d4ef604eeb8edcf0aa22932ca1 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:55.549331 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb6360a-fec0-4641-8c07-978b19586bc2 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:55.567989 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:55.603861 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:55.661726 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771001c3-4b16-4da2-badb-ac07d3aa022b None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:55.668830 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb6360a-fec0-4641-8c07-978b19586bc2 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:55.698606 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e05d7d4-c079-4d70-9838-24e89d0e7431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:55.699290 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e05d7d4-c079-4d70-9838-24e89d0e7431 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:55.699512 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e05d7d4-c079-4d70-9838-24e89d0e7431 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:55.699779 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e05d7d4-c079-4d70-9838-24e89d0e7431 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:55.710412 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1470ac9e-5449-4768-809b-f775b0466afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:55.711069 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1470ac9e-5449-4768-809b-f775b0466afb None admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:55.712102 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1470ac9e-5449-4768-809b-f775b0466afb None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:55.712102 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1470ac9e-5449-4768-809b-f775b0466afb None admin] PATH_INFO: `/v3/projects/4bb6307eb0644d85a1d1daaa61352697/users/e7fc9da7505744608dfd3bbd46a0a9d8/roles/d90649d4ef604eeb8edcf0aa22932ca1` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:55.712323 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e05d7d4-c079-4d70-9838-24e89d0e7431 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:50:55.717343 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-8e05d7d4-c079-4d70-9838-24e89d0e7431 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 21:50:55.717343 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:55.717343 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:55.717343 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:55.717343 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:55.717343 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:55.717343 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:55.717343 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:55.717343 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:55.717343 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:55.717343 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:55.717343 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:55.717343 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:50:55.717343 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:50:55.717343 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:50:55.717343 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:50:55.718356 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:50:55.718356 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:50:55.718356 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:50:55.718356 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:50:55.718356 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:55.718356 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:55.718356 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:55.718356 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:55.718356 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:55.718356 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:55.718356 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:55.718356 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:55.718356 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:55.718356 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:55.718356 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:55.718356 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:55.719662 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:55.719662 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:55.719662 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:55.719662 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:55.719662 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:50:55.719662 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:50:55.719662 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:50:55.719662 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:50:55.719662 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:50:55.719662 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:50:55.719662 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 21:50:55.719662 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:50:55.719662 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 319/646] 10.222.0.18 () {56 vars in 1102 bytes} [Sat Aug 6 21:50:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:55.719662 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e05d7d4-c079-4d70-9838-24e89d0e7431 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:55.735378 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1470ac9e-5449-4768-809b-f775b0466afb None admin] RBAC: Authorizing `identity:create_grant(project_id=4bb6307eb0644d85a1d1daaa61352697, user_id=e7fc9da7505744608dfd3bbd46a0a9d8, role_id=d90649d4ef604eeb8edcf0aa22932ca1)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:55.737584 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1470ac9e-5449-4768-809b-f775b0466afb None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:55.739435 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:55.826206 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e05d7d4-c079-4d70-9838-24e89d0e7431 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:55.830579 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 328/647] 10.222.0.18 () {58 vars in 1439 bytes} [Sat Aug 6 21:50:55 2022] PUT /identity/v3/projects/4bb6307eb0644d85a1d1daaa61352697/users/e7fc9da7505744608dfd3bbd46a0a9d8/roles/d90649d4ef604eeb8edcf0aa22932ca1 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 21:50:55.831652 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1470ac9e-5449-4768-809b-f775b0466afb None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:55.878162 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349eb93a-266e-4d47-8696-d3410f192176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:55.878912 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349eb93a-266e-4d47-8696-d3410f192176 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:55.879199 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349eb93a-266e-4d47-8696-d3410f192176 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:55.879476 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349eb93a-266e-4d47-8696-d3410f192176 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:55.890344 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349eb93a-266e-4d47-8696-d3410f192176 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:55.892725 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349eb93a-266e-4d47-8696-d3410f192176 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:55.910656 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-349eb93a-266e-4d47-8696-d3410f192176 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 06 21:50:55.910656 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:55.910656 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:55.910656 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:55.910656 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:55.910656 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:55.910656 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:55.910656 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:55.910656 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:55.910656 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:55.910656 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:55.910656 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:55.910656 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 21:50:55.910656 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 21:50:55.910656 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 21:50:55.910656 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 21:50:55.911686 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:55.911686 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:55.911686 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 21:50:55.911686 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 21:50:55.911686 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 21:50:55.911686 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 21:50:55.911686 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:55.911686 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:55.911686 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:55.911686 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:55.911686 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:55.911686 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:55.911686 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:55.911686 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:55.911686 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:55.911686 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:55.911686 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:55.911686 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:55.912742 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:55.912742 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:55.912742 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 21:50:55.912742 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 21:50:55.912742 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 21:50:55.912742 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 21:50:55.912742 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 21:50:55.912742 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 21:50:55.912742 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 06 21:50:55.912742 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:50:55.916511 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 320/648] 10.222.0.18 () {56 vars in 1099 bytes} [Sat Aug 6 21:50:55 2022] GET /identity/v3/users/glance => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:55.918220 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349eb93a-266e-4d47-8696-d3410f192176 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:55.969662 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:55.977276 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1470ac9e-5449-4768-809b-f775b0466afb None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:56.023913 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:56.027376 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42922034-5556-48ce-80b7-ad05af5138ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:56.028012 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42922034-5556-48ce-80b7-ad05af5138ef None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:56.028285 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42922034-5556-48ce-80b7-ad05af5138ef None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:56.028604 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42922034-5556-48ce-80b7-ad05af5138ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:56.048062 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42922034-5556-48ce-80b7-ad05af5138ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:56.049573 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42922034-5556-48ce-80b7-ad05af5138ef None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:56.051188 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349eb93a-266e-4d47-8696-d3410f192176 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:56.072020 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-42922034-5556-48ce-80b7-ad05af5138ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:50:56.072020 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:56.072020 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:56.072020 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:56.072020 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:56.072020 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:56.072020 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:56.072020 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:56.072020 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:56.072020 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:56.072020 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:56.072020 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:56.072020 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:50:56.072020 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:50:56.072020 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:50:56.072020 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:50:56.073243 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:56.073243 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:56.073243 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:56.073243 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:56.073243 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:56.073243 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:56.073243 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:56.073243 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:56.073243 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:56.073243 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:56.073243 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:56.073243 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:56.073243 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:56.073243 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:56.073243 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:56.073243 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:56.073243 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:50:56.074317 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:50:56.074317 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:50:56.074317 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:50:56.076805 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 329/649] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:50:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:56.078008 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42922034-5556-48ce-80b7-ad05af5138ef None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:56.095096 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265e1bc2-bb6b-4473-a1a8-00291adcab9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:56.096094 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265e1bc2-bb6b-4473-a1a8-00291adcab9c None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:56.096429 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265e1bc2-bb6b-4473-a1a8-00291adcab9c None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:56.096807 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265e1bc2-bb6b-4473-a1a8-00291adcab9c None admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:56.097579 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265e1bc2-bb6b-4473-a1a8-00291adcab9c None admin] RBAC: Adding query filter params (name=service) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:56.097985 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265e1bc2-bb6b-4473-a1a8-00291adcab9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:56.099466 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265e1bc2-bb6b-4473-a1a8-00291adcab9c None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:56.108654 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 321/650] 10.222.0.18 () {56 vars in 1103 bytes} [Sat Aug 6 21:50:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:56.114758 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265e1bc2-bb6b-4473-a1a8-00291adcab9c None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:56.211088 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42922034-5556-48ce-80b7-ad05af5138ef None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:56.225807 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265e1bc2-bb6b-4473-a1a8-00291adcab9c None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:56.261498 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10bf23c-5c15-4b57-9669-77f65993e538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:56.262107 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10bf23c-5c15-4b57-9669-77f65993e538 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:56.262383 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10bf23c-5c15-4b57-9669-77f65993e538 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:56.262651 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10bf23c-5c15-4b57-9669-77f65993e538 None admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:56.263379 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10bf23c-5c15-4b57-9669-77f65993e538 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:56.263682 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10bf23c-5c15-4b57-9669-77f65993e538 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:56.265145 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10bf23c-5c15-4b57-9669-77f65993e538 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:56.270636 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902eb959-81cd-42a7-b166-d112bf48220d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:56.271527 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902eb959-81cd-42a7-b166-d112bf48220d None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:56.272011 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902eb959-81cd-42a7-b166-d112bf48220d None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:56.272771 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902eb959-81cd-42a7-b166-d112bf48220d None admin] PATH_INFO: `/v3/domains` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:56.273623 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902eb959-81cd-42a7-b166-d112bf48220d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:56.274378 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902eb959-81cd-42a7-b166-d112bf48220d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:56.276936 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902eb959-81cd-42a7-b166-d112bf48220d None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:56.279769 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:56.288102 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 330/651] 10.222.0.18 () {56 vars in 1137 bytes} [Sat Aug 6 21:50:56 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:56.289288 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10bf23c-5c15-4b57-9669-77f65993e538 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:56.297568 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 322/652] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:50:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:56.299241 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:56.302863 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902eb959-81cd-42a7-b166-d112bf48220d None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:56.417842 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10bf23c-5c15-4b57-9669-77f65993e538 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:56.437160 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902eb959-81cd-42a7-b166-d112bf48220d None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:56.455129 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe35b683-985a-4d41-89f4-9cd23b0b65c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:56.455829 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe35b683-985a-4d41-89f4-9cd23b0b65c6 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:56.456165 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe35b683-985a-4d41-89f4-9cd23b0b65c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:56.456534 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe35b683-985a-4d41-89f4-9cd23b0b65c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:56.467341 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe35b683-985a-4d41-89f4-9cd23b0b65c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:56.468989 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe35b683-985a-4d41-89f4-9cd23b0b65c6 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:56.485085 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-fe35b683-985a-4d41-89f4-9cd23b0b65c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:50:56.485085 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:56.485085 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:56.485085 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:56.485085 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:56.485085 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:56.485085 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:56.485085 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:56.485085 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:56.485085 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:56.485085 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:56.485085 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:56.485085 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:50:56.485085 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:50:56.485085 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:50:56.485085 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:50:56.486343 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:56.486343 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:56.486343 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:56.486343 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:56.486343 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:56.486343 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:56.486343 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:56.486343 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:56.486343 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:56.486343 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:56.486343 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:56.486343 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:56.486343 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:56.486343 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:56.486343 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:56.486343 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:56.486343 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:50:56.487261 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:50:56.487261 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:50:56.487261 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:50:56.487261 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 331/653] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:50:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:56.487487 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe35b683-985a-4d41-89f4-9cd23b0b65c6 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:56.490243 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f86e14-50e4-48f6-86df-a8681a051be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:56.491344 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f86e14-50e4-48f6-86df-a8681a051be2 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:56.492610 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f86e14-50e4-48f6-86df-a8681a051be2 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:56.492907 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f86e14-50e4-48f6-86df-a8681a051be2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:56.558045 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:56.577113 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:56.587785 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f86e14-50e4-48f6-86df-a8681a051be2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:56.589689 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f86e14-50e4-48f6-86df-a8681a051be2 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:56.606305 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe35b683-985a-4d41-89f4-9cd23b0b65c6 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:56.645513 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6d4586-0dff-4d52-9ff4-f4a62449e949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:56.646160 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d4586-0dff-4d52-9ff4-f4a62449e949 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:56.646432 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d4586-0dff-4d52-9ff4-f4a62449e949 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:56.646697 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d4586-0dff-4d52-9ff4-f4a62449e949 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:56.654762 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6d4586-0dff-4d52-9ff4-f4a62449e949 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:56.656124 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6d4586-0dff-4d52-9ff4-f4a62449e949 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:56.667797 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-6f6d4586-0dff-4d52-9ff4-f4a62449e949 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 06 21:50:56.667797 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:56.667797 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:56.667797 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:56.667797 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:56.667797 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:56.667797 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:56.667797 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:56.667797 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:56.667797 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:56.667797 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:56.667797 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:56.667797 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 21:50:56.667797 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 21:50:56.667797 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 21:50:56.667797 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 21:50:56.668830 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:56.668830 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:56.668830 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 21:50:56.668830 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 21:50:56.668830 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 21:50:56.668830 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 21:50:56.668830 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:56.668830 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:56.668830 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:56.668830 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:56.668830 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:56.668830 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:56.668830 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:56.668830 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:56.668830 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:56.668830 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:56.668830 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:56.668830 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:56.669785 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:56.669785 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:56.669785 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 21:50:56.669785 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 21:50:56.669785 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 21:50:56.669785 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 21:50:56.669785 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 21:50:56.669785 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 21:50:56.669785 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 06 21:50:56.669785 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:50:56.669785 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 332/654] 10.222.0.18 () {56 vars in 1099 bytes} [Sat Aug 6 21:50:56 2022] GET /identity/v3/users/cinder => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:56.670402 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6d4586-0dff-4d52-9ff4-f4a62449e949 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:56.723385 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 323/655] 10.222.0.18 () {58 vars in 1282 bytes} [Sat Aug 6 21:50:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 21:50:56.724231 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f86e14-50e4-48f6-86df-a8681a051be2 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:56.793170 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6d4586-0dff-4d52-9ff4-f4a62449e949 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:56.814324 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:56.827470 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:56.833653 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f86e14-50e4-48f6-86df-a8681a051be2 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:56.834042 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2345cc-1b23-4d5a-9cb3-7bd91b055268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:56.834859 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2345cc-1b23-4d5a-9cb3-7bd91b055268 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:56.835303 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2345cc-1b23-4d5a-9cb3-7bd91b055268 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:56.835737 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2345cc-1b23-4d5a-9cb3-7bd91b055268 None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:56.836565 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2345cc-1b23-4d5a-9cb3-7bd91b055268 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:56.837030 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2345cc-1b23-4d5a-9cb3-7bd91b055268 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:56.838635 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2345cc-1b23-4d5a-9cb3-7bd91b055268 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:56.852510 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 333/656] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:50:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:56.853739 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2345cc-1b23-4d5a-9cb3-7bd91b055268 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:56.871317 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d62055-813c-4a2c-a881-86be1b59b7b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:56.871742 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d62055-813c-4a2c-a881-86be1b59b7b9 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:56.872024 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d62055-813c-4a2c-a881-86be1b59b7b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:56.872303 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d62055-813c-4a2c-a881-86be1b59b7b9 None admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:56.873153 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d62055-813c-4a2c-a881-86be1b59b7b9 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:56.873399 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d62055-813c-4a2c-a881-86be1b59b7b9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:56.875513 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d62055-813c-4a2c-a881-86be1b59b7b9 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:56.896133 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 324/657] 10.222.0.18 () {56 vars in 1137 bytes} [Sat Aug 6 21:50:56 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:56.897087 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d62055-813c-4a2c-a881-86be1b59b7b9 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:56.968549 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2345cc-1b23-4d5a-9cb3-7bd91b055268 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:57.005009 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d62055-813c-4a2c-a881-86be1b59b7b9 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:57.008668 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f734e6-7c33-4ba8-a6da-92a35dda191d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:57.009617 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f734e6-7c33-4ba8-a6da-92a35dda191d None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:57.010063 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f734e6-7c33-4ba8-a6da-92a35dda191d None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:57.010486 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f734e6-7c33-4ba8-a6da-92a35dda191d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:57.022895 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f734e6-7c33-4ba8-a6da-92a35dda191d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:57.024918 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f734e6-7c33-4ba8-a6da-92a35dda191d None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:57.042267 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-09f734e6-7c33-4ba8-a6da-92a35dda191d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:50:57.042267 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:57.042267 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:57.042267 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:57.042267 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:57.042267 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:57.042267 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:57.042267 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:57.042267 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:57.042267 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:57.042267 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:57.042267 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:57.042267 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:50:57.042267 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:50:57.042267 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:50:57.042267 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:50:57.043462 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:57.043462 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:57.043462 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:57.043462 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:57.043462 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:57.043462 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:57.043462 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:57.043462 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:57.043462 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:57.043462 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:57.043462 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:57.043462 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:57.043462 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:57.043462 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:57.043462 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:57.043462 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:57.043462 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:50:57.044633 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:50:57.044633 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:50:57.044633 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:50:57.044829 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cd722c-d411-4ba6-ab2b-2a7ecbcd04ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:57.044829 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cd722c-d411-4ba6-ab2b-2a7ecbcd04ad None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:57.044829 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cd722c-d411-4ba6-ab2b-2a7ecbcd04ad None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:57.044829 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cd722c-d411-4ba6-ab2b-2a7ecbcd04ad None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:57.045617 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 334/658] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:50:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:57.046414 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f734e6-7c33-4ba8-a6da-92a35dda191d None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:57.055277 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cd722c-d411-4ba6-ab2b-2a7ecbcd04ad None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:57.056759 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cd722c-d411-4ba6-ab2b-2a7ecbcd04ad None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:57.069950 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-39cd722c-d411-4ba6-ab2b-2a7ecbcd04ad None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 06 21:50:57.069950 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:57.069950 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:57.069950 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:57.069950 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:57.069950 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:57.069950 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:57.069950 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:57.069950 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:57.069950 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:57.069950 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:57.069950 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:57.069950 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 21:50:57.069950 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 21:50:57.069950 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 21:50:57.069950 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 21:50:57.070912 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:57.070912 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:57.070912 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 21:50:57.070912 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 21:50:57.070912 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 21:50:57.070912 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 21:50:57.070912 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:57.070912 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:57.070912 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:57.070912 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:57.070912 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:57.070912 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:57.070912 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:57.070912 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:57.070912 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:57.070912 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:57.070912 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:57.070912 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:57.071905 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:57.071905 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:57.071905 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 21:50:57.071905 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 21:50:57.071905 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 21:50:57.071905 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 21:50:57.071905 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 21:50:57.071905 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 21:50:57.071905 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 06 21:50:57.071905 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:50:57.071905 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 325/659] 10.222.0.18 () {56 vars in 1102 bytes} [Sat Aug 6 21:50:56 2022] GET /identity/v3/users/neutron => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:57.072557 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cd722c-d411-4ba6-ab2b-2a7ecbcd04ad None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:57.084565 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:57.097958 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:57.163823 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f734e6-7c33-4ba8-a6da-92a35dda191d None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:57.213341 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aad324-8b9b-4486-94e5-761b658ad143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:57.213858 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cd722c-d411-4ba6-ab2b-2a7ecbcd04ad None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:57.214558 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aad324-8b9b-4486-94e5-761b658ad143 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:57.215040 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aad324-8b9b-4486-94e5-761b658ad143 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:57.215736 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aad324-8b9b-4486-94e5-761b658ad143 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:57.253937 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab741da-7e26-48ba-89c1-af9671d32ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:57.254553 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab741da-7e26-48ba-89c1-af9671d32ce7 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:57.254836 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab741da-7e26-48ba-89c1-af9671d32ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:57.255167 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab741da-7e26-48ba-89c1-af9671d32ce7 None admin] PATH_INFO: `/v3/domains` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:57.255818 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab741da-7e26-48ba-89c1-af9671d32ce7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:57.256155 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab741da-7e26-48ba-89c1-af9671d32ce7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:57.257819 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab741da-7e26-48ba-89c1-af9671d32ce7 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:57.271465 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 326/660] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:50:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:57.272367 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab741da-7e26-48ba-89c1-af9671d32ce7 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:57.314830 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04aad324-8b9b-4486-94e5-761b658ad143 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:57.317218 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04aad324-8b9b-4486-94e5-761b658ad143 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:57.364395 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:57.366319 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:57.392838 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab741da-7e26-48ba-89c1-af9671d32ce7 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:57.429593 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5063528-65aa-4e8f-bd42-eb9dc33f0aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:57.430208 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5063528-65aa-4e8f-bd42-eb9dc33f0aef None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:57.430489 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5063528-65aa-4e8f-bd42-eb9dc33f0aef None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:57.430763 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5063528-65aa-4e8f-bd42-eb9dc33f0aef None admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:57.431503 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5063528-65aa-4e8f-bd42-eb9dc33f0aef None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:57.431862 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5063528-65aa-4e8f-bd42-eb9dc33f0aef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:57.433363 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5063528-65aa-4e8f-bd42-eb9dc33f0aef None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:57.450352 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 335/661] 10.222.0.18 () {58 vars in 1282 bytes} [Sat Aug 6 21:50:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 21:50:57.451511 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aad324-8b9b-4486-94e5-761b658ad143 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:57.452375 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 327/662] 10.222.0.18 () {56 vars in 1139 bytes} [Sat Aug 6 21:50:57 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:57.461425 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5063528-65aa-4e8f-bd42-eb9dc33f0aef None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:57.566131 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5063528-65aa-4e8f-bd42-eb9dc33f0aef None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:57.568273 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aad324-8b9b-4486-94e5-761b658ad143 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:57.596865 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:57.601680 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38c4653-8171-4e20-9280-35213d7e3f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:57.602323 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38c4653-8171-4e20-9280-35213d7e3f2e None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:57.602603 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38c4653-8171-4e20-9280-35213d7e3f2e None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:57.602869 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38c4653-8171-4e20-9280-35213d7e3f2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:57.604105 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:57.611017 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb4191a-3b04-47be-a852-d4988591630c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:57.611953 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb4191a-3b04-47be-a852-d4988591630c None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:57.612591 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb4191a-3b04-47be-a852-d4988591630c None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:57.613110 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb4191a-3b04-47be-a852-d4988591630c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:57.614896 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38c4653-8171-4e20-9280-35213d7e3f2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:57.616610 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38c4653-8171-4e20-9280-35213d7e3f2e None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:57.621616 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb4191a-3b04-47be-a852-d4988591630c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:57.623823 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb4191a-3b04-47be-a852-d4988591630c None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:57.632857 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-a38c4653-8171-4e20-9280-35213d7e3f2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:50:57.632857 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:57.632857 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:57.632857 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:57.632857 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:57.632857 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:57.632857 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:57.632857 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:57.632857 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:57.632857 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:57.632857 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:57.632857 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:57.632857 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:50:57.632857 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:50:57.632857 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:50:57.632857 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:50:57.634042 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:57.634042 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:57.634042 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:57.634042 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:57.634042 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:57.634042 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:57.634042 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:57.634042 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:57.634042 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:57.634042 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:57.634042 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:57.634042 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:57.634042 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:57.634042 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:57.634042 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:57.634042 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:57.634042 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:50:57.635205 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:50:57.635205 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:50:57.635205 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:50:57.637202 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 328/663] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:50:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:57.638302 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38c4653-8171-4e20-9280-35213d7e3f2e None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:57.642368 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-efb4191a-3b04-47be-a852-d4988591630c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 21:50:57.642368 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:57.642368 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:57.642368 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:57.642368 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:57.642368 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:57.642368 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:57.642368 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:57.642368 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:57.642368 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:57.642368 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:57.642368 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:57.642368 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 21:50:57.642368 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 21:50:57.642368 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 21:50:57.643443 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 21:50:57.643443 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:57.643443 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:57.643443 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:57.643443 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:57.643443 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:57.643443 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:57.643443 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:57.643443 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:57.643443 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:57.643443 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:57.643443 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:57.643443 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:57.643443 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:57.643443 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:57.643443 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:57.643443 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:57.644591 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 21:50:57.644591 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 21:50:57.644591 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 21:50:57.644591 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 21:50:57.644591 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 21:50:57.644591 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 21:50:57.644591 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 21:50:57.644591 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:50:57.647687 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 336/664] 10.222.0.18 () {56 vars in 1111 bytes} [Sat Aug 6 21:50:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:57.648681 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb4191a-3b04-47be-a852-d4988591630c None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:57.757724 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38c4653-8171-4e20-9280-35213d7e3f2e None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:57.764008 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb4191a-3b04-47be-a852-d4988591630c None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:57.795323 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48620918-0b00-45a7-acde-b70b2f109529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:57.795933 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48620918-0b00-45a7-acde-b70b2f109529 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:57.796222 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48620918-0b00-45a7-acde-b70b2f109529 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:57.796540 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48620918-0b00-45a7-acde-b70b2f109529 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:57.800743 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9781ae65-d2b0-447b-aca0-2e45733384cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:57.801340 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9781ae65-d2b0-447b-aca0-2e45733384cd None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:57.801616 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9781ae65-d2b0-447b-aca0-2e45733384cd None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:57.801882 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9781ae65-d2b0-447b-aca0-2e45733384cd None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:57.802517 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9781ae65-d2b0-447b-aca0-2e45733384cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:57.802812 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9781ae65-d2b0-447b-aca0-2e45733384cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:57.804197 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9781ae65-d2b0-447b-aca0-2e45733384cd None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:57.820743 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 337/665] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:50:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:57.821486 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9781ae65-d2b0-447b-aca0-2e45733384cd None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:57.853006 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:57.870774 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48620918-0b00-45a7-acde-b70b2f109529 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:57.872827 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48620918-0b00-45a7-acde-b70b2f109529 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:57.876772 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:57.929875 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9781ae65-d2b0-447b-aca0-2e45733384cd None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:57.966697 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d303720f-9f65-4d0c-a7e8-8bf1da99dcc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:57.967299 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d303720f-9f65-4d0c-a7e8-8bf1da99dcc2 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:57.967577 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d303720f-9f65-4d0c-a7e8-8bf1da99dcc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:57.967847 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d303720f-9f65-4d0c-a7e8-8bf1da99dcc2 None admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:57.968588 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d303720f-9f65-4d0c-a7e8-8bf1da99dcc2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:57.968946 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d303720f-9f65-4d0c-a7e8-8bf1da99dcc2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:57.970362 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d303720f-9f65-4d0c-a7e8-8bf1da99dcc2 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:57.975517 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 329/666] 10.222.0.18 () {58 vars in 1282 bytes} [Sat Aug 6 21:50:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 21:50:57.976768 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee00e51-00c4-44fc-87dd-196010179040 None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:57.977048 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee00e51-00c4-44fc-87dd-196010179040 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:57.977322 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee00e51-00c4-44fc-87dd-196010179040 None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:57.978536 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 330/667] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:50:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:50:57.979224 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee00e51-00c4-44fc-87dd-196010179040 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:57.983932 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 338/668] 10.222.0.18 () {56 vars in 1148 bytes} [Sat Aug 6 21:50:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:57.985901 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2f7db7-ee19-45ef-a48e-25cfebf0f6da None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:57.986182 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2f7db7-ee19-45ef-a48e-25cfebf0f6da None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:57.986457 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2f7db7-ee19-45ef-a48e-25cfebf0f6da None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:57.987769 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 339/669] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:50:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:50:57.988318 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2f7db7-ee19-45ef-a48e-25cfebf0f6da None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:58.069197 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:58.085381 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee00e51-00c4-44fc-87dd-196010179040 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:58.088269 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2f7db7-ee19-45ef-a48e-25cfebf0f6da None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:58.098853 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:58.119203 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3347ad1-29c4-45ba-91cf-db000c3ee83f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:58.119821 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3347ad1-29c4-45ba-91cf-db000c3ee83f None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:58.120097 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3347ad1-29c4-45ba-91cf-db000c3ee83f None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:58.120365 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3347ad1-29c4-45ba-91cf-db000c3ee83f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:58.121936 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522861d5-dc2b-4e9c-9334-01f61fdf2e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:58.122549 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522861d5-dc2b-4e9c-9334-01f61fdf2e38 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:58.122836 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522861d5-dc2b-4e9c-9334-01f61fdf2e38 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:58.123100 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522861d5-dc2b-4e9c-9334-01f61fdf2e38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:58.127954 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3347ad1-29c4-45ba-91cf-db000c3ee83f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:58.129418 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3347ad1-29c4-45ba-91cf-db000c3ee83f None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:58.133765 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522861d5-dc2b-4e9c-9334-01f61fdf2e38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:58.135209 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522861d5-dc2b-4e9c-9334-01f61fdf2e38 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:58.140472 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-d3347ad1-29c4-45ba-91cf-db000c3ee83f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 21:50:58.140472 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:58.140472 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:58.140472 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:58.140472 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:58.140472 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:58.140472 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:58.140472 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:58.140472 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:58.140472 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:58.140472 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:58.140472 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:58.140472 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 21:50:58.140472 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 21:50:58.140472 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 21:50:58.142527 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 21:50:58.142527 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:58.142527 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:58.142527 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:58.142527 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:58.142527 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:58.142527 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:58.142527 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:58.142527 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:58.142527 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:58.142527 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:58.142527 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:58.142527 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:58.142527 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:58.142527 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:58.142527 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:58.142527 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:58.143418 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 21:50:58.143418 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 21:50:58.143418 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 21:50:58.143418 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 21:50:58.143418 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 21:50:58.143418 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 21:50:58.143418 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 21:50:58.143418 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:50:58.143418 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 331/670] 10.222.0.18 () {56 vars in 1111 bytes} [Sat Aug 6 21:50:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:58.143418 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0facc486-8627-46bf-8579-320e44b9f4b0 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:58.143418 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0facc486-8627-46bf-8579-320e44b9f4b0 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:58.144058 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0facc486-8627-46bf-8579-320e44b9f4b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:58.150706 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-522861d5-dc2b-4e9c-9334-01f61fdf2e38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:50:58.150706 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:58.150706 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:58.150706 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:58.150706 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:58.150706 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:58.150706 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:58.150706 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:58.150706 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:58.150706 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:58.150706 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:58.150706 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:58.150706 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:50:58.150706 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:50:58.150706 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:50:58.150706 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:50:58.151697 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:58.151697 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:58.151697 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:58.151697 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:58.151697 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:58.151697 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:58.151697 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:58.151697 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:58.151697 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:58.151697 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:58.151697 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:58.151697 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:58.151697 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:58.151697 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:58.151697 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:58.151697 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:58.151697 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:50:58.152627 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:50:58.152627 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:50:58.152627 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:50:58.152627 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 340/671] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:50:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:58.152627 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522861d5-dc2b-4e9c-9334-01f61fdf2e38 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:58.162129 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-0facc486-8627-46bf-8579-320e44b9f4b0 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:50:58.163314 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-0facc486-8627-46bf-8579-320e44b9f4b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:50:58.228016 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 332/672] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:50:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 21:50:58.229625 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f40a75-3df3-4738-a78a-52df82ba13b6 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:58.229911 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f40a75-3df3-4738-a78a-52df82ba13b6 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:58.230177 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f40a75-3df3-4738-a78a-52df82ba13b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:58.249157 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-c6f40a75-3df3-4738-a78a-52df82ba13b6 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:50:58.250959 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-c6f40a75-3df3-4738-a78a-52df82ba13b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:50:58.259821 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522861d5-dc2b-4e9c-9334-01f61fdf2e38 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:58.292371 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dbe884-c3d3-4497-98b4-6f0d48ec61d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:58.292996 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dbe884-c3d3-4497-98b4-6f0d48ec61d8 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:58.293276 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dbe884-c3d3-4497-98b4-6f0d48ec61d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:58.293546 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dbe884-c3d3-4497-98b4-6f0d48ec61d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:58.294259 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dbe884-c3d3-4497-98b4-6f0d48ec61d8 None admin] RBAC: Adding query filter params (role.id=d90649d4ef604eeb8edcf0aa22932ca1, scope.project.id=4bb6307eb0644d85a1d1daaa61352697, user.id=c8d38e73e105476e8de858cfa939ba20) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:58.294562 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dbe884-c3d3-4497-98b4-6f0d48ec61d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:58.295990 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dbe884-c3d3-4497-98b4-6f0d48ec61d8 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:58.302146 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 341/673] 10.222.0.18 () {56 vars in 1374 bytes} [Sat Aug 6 21:50:58 2022] GET /identity/v3/role_assignments?user.id=c8d38e73e105476e8de858cfa939ba20&scope.project.id=4bb6307eb0644d85a1d1daaa61352697&role.id=d90649d4ef604eeb8edcf0aa22932ca1 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:58.302905 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dbe884-c3d3-4497-98b4-6f0d48ec61d8 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:58.332651 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 333/674] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:50:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 21:50:58.332651 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f40a75-3df3-4738-a78a-52df82ba13b6 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:58.333817 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:58.338417 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:58.423713 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dbe884-c3d3-4497-98b4-6f0d48ec61d8 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:58.458066 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f40a75-3df3-4738-a78a-52df82ba13b6 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:58.462934 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862381a8-5a59-404f-b275-69be256b74d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:58.462934 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862381a8-5a59-404f-b275-69be256b74d2 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:58.463159 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862381a8-5a59-404f-b275-69be256b74d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:58.463357 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862381a8-5a59-404f-b275-69be256b74d2 None admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:58.464372 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862381a8-5a59-404f-b275-69be256b74d2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:58.464372 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862381a8-5a59-404f-b275-69be256b74d2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:58.466644 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862381a8-5a59-404f-b275-69be256b74d2 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:58.480747 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 342/675] 10.222.0.18 () {56 vars in 1148 bytes} [Sat Aug 6 21:50:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:58.481973 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862381a8-5a59-404f-b275-69be256b74d2 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:58.497795 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9677113d-5ffe-42ed-8e17-2ef65dd80d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:58.498407 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9677113d-5ffe-42ed-8e17-2ef65dd80d37 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:58.498684 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9677113d-5ffe-42ed-8e17-2ef65dd80d37 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:58.498976 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9677113d-5ffe-42ed-8e17-2ef65dd80d37 None admin] PATH_INFO: `/v3/domains` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:58.499591 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9677113d-5ffe-42ed-8e17-2ef65dd80d37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:58.500516 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9677113d-5ffe-42ed-8e17-2ef65dd80d37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:58.501893 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9677113d-5ffe-42ed-8e17-2ef65dd80d37 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:58.514731 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 334/676] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:50:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:58.515613 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9677113d-5ffe-42ed-8e17-2ef65dd80d37 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:58.583255 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:58.583881 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:58.599192 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862381a8-5a59-404f-b275-69be256b74d2 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:58.624628 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9677113d-5ffe-42ed-8e17-2ef65dd80d37 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:58.636628 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c666ec38-2b31-40bf-89c1-a39e0e65b2d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:58.637511 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c666ec38-2b31-40bf-89c1-a39e0e65b2d6 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:58.637971 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c666ec38-2b31-40bf-89c1-a39e0e65b2d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:58.638245 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c666ec38-2b31-40bf-89c1-a39e0e65b2d6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:58.648742 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c666ec38-2b31-40bf-89c1-a39e0e65b2d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:50:58.653285 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-c666ec38-2b31-40bf-89c1-a39e0e65b2d6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 21:50:58.653285 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:58.653285 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:58.653285 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:58.653285 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:58.653285 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:58.653285 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:58.653285 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:58.653285 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:58.653285 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:58.653285 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:58.653285 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:58.653285 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:50:58.653285 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:50:58.653285 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:50:58.653285 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:50:58.654275 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:50:58.654275 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:50:58.654275 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:50:58.654275 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:50:58.654275 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:58.654275 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:58.654275 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:58.654275 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:58.654275 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:58.654275 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:58.654275 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:58.654275 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:58.654275 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:58.654275 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:58.654275 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:58.654275 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:58.655117 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:58.655117 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:58.655117 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:58.655117 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:58.655117 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:50:58.655117 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:50:58.655117 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:50:58.655117 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:50:58.655117 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:50:58.655117 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:50:58.655117 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 21:50:58.655117 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:50:58.655117 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 343/677] 10.222.0.18 () {56 vars in 1096 bytes} [Sat Aug 6 21:50:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:58.655117 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c666ec38-2b31-40bf-89c1-a39e0e65b2d6 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:58.671924 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf3ec32-d97f-4c91-8543-c429e8f11fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:58.672507 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf3ec32-d97f-4c91-8543-c429e8f11fc6 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:58.672784 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf3ec32-d97f-4c91-8543-c429e8f11fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:58.673052 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf3ec32-d97f-4c91-8543-c429e8f11fc6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:58.683771 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf3ec32-d97f-4c91-8543-c429e8f11fc6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:50:58.689987 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-bdf3ec32-d97f-4c91-8543-c429e8f11fc6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 21:50:58.689987 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:58.689987 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:58.689987 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:58.689987 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:58.689987 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:58.689987 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:58.689987 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:58.689987 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:58.689987 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:58.689987 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:58.689987 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:58.689987 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:50:58.689987 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:50:58.689987 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:50:58.689987 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:50:58.691861 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:50:58.691861 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:50:58.691861 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:50:58.691861 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:50:58.691861 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:58.691861 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:58.691861 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:58.691861 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:58.691861 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:58.691861 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:58.691861 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:58.691861 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:58.691861 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:58.691861 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:58.691861 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:58.691861 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:58.693548 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:58.693548 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:58.693548 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:58.693548 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:58.693548 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:50:58.693548 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:50:58.693548 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:50:58.693548 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:50:58.693548 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:50:58.693548 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:50:58.693548 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 21:50:58.693548 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:50:58.701825 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 335/678] 10.222.0.18 () {56 vars in 1102 bytes} [Sat Aug 6 21:50:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:58.701825 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf3ec32-d97f-4c91-8543-c429e8f11fc6 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:58.763159 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c666ec38-2b31-40bf-89c1-a39e0e65b2d6 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:58.800162 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248530b1-0451-4bd3-b407-eacc233b73b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:58.800904 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248530b1-0451-4bd3-b407-eacc233b73b0 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:58.801181 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248530b1-0451-4bd3-b407-eacc233b73b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:58.801480 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248530b1-0451-4bd3-b407-eacc233b73b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:58.802262 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248530b1-0451-4bd3-b407-eacc233b73b0 None admin] RBAC: Adding query filter params (role.id=d90649d4ef604eeb8edcf0aa22932ca1, scope.project.id=4bb6307eb0644d85a1d1daaa61352697, user.id=aa71c71de263478da234f3c5716a6e5f) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:58.802573 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248530b1-0451-4bd3-b407-eacc233b73b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:58.804111 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248530b1-0451-4bd3-b407-eacc233b73b0 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:58.807380 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf3ec32-d97f-4c91-8543-c429e8f11fc6 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:58.809995 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 344/679] 10.222.0.18 () {56 vars in 1374 bytes} [Sat Aug 6 21:50:58 2022] GET /identity/v3/role_assignments?user.id=aa71c71de263478da234f3c5716a6e5f&scope.project.id=4bb6307eb0644d85a1d1daaa61352697&role.id=d90649d4ef604eeb8edcf0aa22932ca1 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:58.810777 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248530b1-0451-4bd3-b407-eacc233b73b0 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:58.824889 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:58.844900 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:58.862163 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0b1e7b-039a-437f-9efa-c7bd36da5cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:58.862746 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0b1e7b-039a-437f-9efa-c7bd36da5cf8 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:58.863026 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0b1e7b-039a-437f-9efa-c7bd36da5cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:58.863623 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0b1e7b-039a-437f-9efa-c7bd36da5cf8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:58.873379 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0b1e7b-039a-437f-9efa-c7bd36da5cf8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:58.876156 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0b1e7b-039a-437f-9efa-c7bd36da5cf8 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:58.891683 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-7b0b1e7b-039a-437f-9efa-c7bd36da5cf8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 21:50:58.891683 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:58.891683 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:58.891683 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:58.891683 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:58.891683 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:58.891683 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:58.891683 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:58.891683 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:58.891683 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:58.891683 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:58.891683 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:58.891683 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 21:50:58.891683 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 21:50:58.891683 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 21:50:58.892681 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 21:50:58.892681 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:58.892681 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:58.892681 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:58.892681 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:58.892681 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:58.892681 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:58.892681 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:58.892681 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:58.892681 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:58.892681 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:58.892681 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:58.892681 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:58.892681 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:58.892681 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:58.892681 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:58.892681 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:58.893593 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 21:50:58.893593 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 21:50:58.893593 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 21:50:58.893593 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 21:50:58.893593 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 21:50:58.893593 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 21:50:58.893593 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 21:50:58.893593 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:50:58.893593 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 336/680] 10.222.0.18 () {56 vars in 1111 bytes} [Sat Aug 6 21:50:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:58.894083 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0b1e7b-039a-437f-9efa-c7bd36da5cf8 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:58.940564 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248530b1-0451-4bd3-b407-eacc233b73b0 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:58.982697 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b20a7bd-039d-4bf2-a6f7-a68c71e4b71f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:58.983304 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b20a7bd-039d-4bf2-a6f7-a68c71e4b71f None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:58.983581 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b20a7bd-039d-4bf2-a6f7-a68c71e4b71f None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:58.983847 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b20a7bd-039d-4bf2-a6f7-a68c71e4b71f None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:58.985084 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b20a7bd-039d-4bf2-a6f7-a68c71e4b71f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:58.986062 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b20a7bd-039d-4bf2-a6f7-a68c71e4b71f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:58.986746 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b20a7bd-039d-4bf2-a6f7-a68c71e4b71f None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:59.000583 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 345/681] 10.222.0.18 () {56 vars in 1099 bytes} [Sat Aug 6 21:50:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:59.000583 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b20a7bd-039d-4bf2-a6f7-a68c71e4b71f None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:59.038185 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0b1e7b-039a-437f-9efa-c7bd36da5cf8 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:59.080611 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:59.082142 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022f6310-ce43-4fd7-af3f-f84ee832295b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:59.082755 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f6310-ce43-4fd7-af3f-f84ee832295b None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:59.083030 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f6310-ce43-4fd7-af3f-f84ee832295b None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:59.083298 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f6310-ce43-4fd7-af3f-f84ee832295b None admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:59.083966 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022f6310-ce43-4fd7-af3f-f84ee832295b None admin] RBAC: Adding query filter params (name=service) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:59.084271 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022f6310-ce43-4fd7-af3f-f84ee832295b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:59.085673 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022f6310-ce43-4fd7-af3f-f84ee832295b None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:59.096799 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 337/682] 10.222.0.18 () {56 vars in 1103 bytes} [Sat Aug 6 21:50:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:59.102957 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022f6310-ce43-4fd7-af3f-f84ee832295b None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:59.116172 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b20a7bd-039d-4bf2-a6f7-a68c71e4b71f None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:59.151213 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:59.153006 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3193bf-413b-4681-9a5c-dbf891dca200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:59.153920 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3193bf-413b-4681-9a5c-dbf891dca200 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:59.154480 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3193bf-413b-4681-9a5c-dbf891dca200 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:59.154943 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3193bf-413b-4681-9a5c-dbf891dca200 None admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:59.155820 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3193bf-413b-4681-9a5c-dbf891dca200 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:59.156896 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3193bf-413b-4681-9a5c-dbf891dca200 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:59.158939 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3193bf-413b-4681-9a5c-dbf891dca200 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:59.173791 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 346/683] 10.222.0.18 () {56 vars in 1148 bytes} [Sat Aug 6 21:50:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:59.177532 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3193bf-413b-4681-9a5c-dbf891dca200 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:59.213360 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022f6310-ce43-4fd7-af3f-f84ee832295b None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:59.247302 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d87083-37a3-484f-86a0-5970afc90a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:59.247980 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d87083-37a3-484f-86a0-5970afc90a72 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:59.248257 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d87083-37a3-484f-86a0-5970afc90a72 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:59.248558 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d87083-37a3-484f-86a0-5970afc90a72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:59.259984 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d87083-37a3-484f-86a0-5970afc90a72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:59.261463 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d87083-37a3-484f-86a0-5970afc90a72 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:59.277359 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-e9d87083-37a3-484f-86a0-5970afc90a72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:50:59.277359 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:59.277359 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:59.277359 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:59.277359 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:59.277359 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:59.277359 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:59.277359 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:59.277359 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:59.277359 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:59.277359 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:59.277359 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:59.277359 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:50:59.277359 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:50:59.277359 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:50:59.277359 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:50:59.278362 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:59.278362 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:59.278362 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:59.278362 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:59.278362 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:59.278362 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:59.278362 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:59.278362 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:59.278362 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:59.278362 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:59.278362 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:59.278362 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:59.278362 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:59.278362 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:59.278362 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:59.278362 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:59.278362 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:50:59.279246 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:50:59.279246 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:50:59.279246 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:50:59.279246 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 338/684] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:50:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:59.279899 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d87083-37a3-484f-86a0-5970afc90a72 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:59.291033 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3193bf-413b-4681-9a5c-dbf891dca200 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:59.323278 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:59.329351 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3903a192-8435-4aa1-a2db-73b2391b9b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:59.330688 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3903a192-8435-4aa1-a2db-73b2391b9b04 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:59.331194 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3903a192-8435-4aa1-a2db-73b2391b9b04 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:59.331661 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3903a192-8435-4aa1-a2db-73b2391b9b04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:59.345470 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3903a192-8435-4aa1-a2db-73b2391b9b04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:59.348691 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3903a192-8435-4aa1-a2db-73b2391b9b04 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:59.370643 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-3903a192-8435-4aa1-a2db-73b2391b9b04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:50:59.370643 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:59.370643 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:59.370643 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:59.370643 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:59.370643 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:59.370643 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:59.370643 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:59.370643 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:59.370643 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:59.370643 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:59.370643 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:59.370643 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:50:59.370643 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:50:59.370643 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:50:59.370643 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:50:59.371599 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:59.371599 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:59.371599 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:59.371599 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:59.371599 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:59.371599 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:59.371599 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:59.371599 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:59.371599 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:59.371599 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:59.371599 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:59.371599 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:59.371599 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:59.371599 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:59.371599 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:59.371599 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:59.371599 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:50:59.372540 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:50:59.372540 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:50:59.372540 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:50:59.379175 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 347/685] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:50:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:59.379175 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3903a192-8435-4aa1-a2db-73b2391b9b04 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:59.388636 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:59.393771 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d87083-37a3-484f-86a0-5970afc90a72 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:59.443393 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b643571-aa24-4b7b-93ff-2183bf3af9d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:59.444586 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b643571-aa24-4b7b-93ff-2183bf3af9d9 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:59.444947 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b643571-aa24-4b7b-93ff-2183bf3af9d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:59.445545 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b643571-aa24-4b7b-93ff-2183bf3af9d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:59.446615 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b643571-aa24-4b7b-93ff-2183bf3af9d9 None admin] RBAC: Adding query filter params (role.id=d90649d4ef604eeb8edcf0aa22932ca1, scope.project.id=4bb6307eb0644d85a1d1daaa61352697, user.id=5ceaeadc843c4186b25114b28607fe22) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:59.447019 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b643571-aa24-4b7b-93ff-2183bf3af9d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:59.449240 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b643571-aa24-4b7b-93ff-2183bf3af9d9 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:59.460494 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 339/686] 10.222.0.18 () {56 vars in 1374 bytes} [Sat Aug 6 21:50:59 2022] GET /identity/v3/role_assignments?user.id=5ceaeadc843c4186b25114b28607fe22&scope.project.id=4bb6307eb0644d85a1d1daaa61352697&role.id=d90649d4ef604eeb8edcf0aa22932ca1 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:59.461485 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b643571-aa24-4b7b-93ff-2183bf3af9d9 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:59.534047 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3903a192-8435-4aa1-a2db-73b2391b9b04 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:59.606566 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e37cbbe-fc51-44f5-b980-5347fa125a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:59.608275 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e37cbbe-fc51-44f5-b980-5347fa125a46 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:59.608840 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e37cbbe-fc51-44f5-b980-5347fa125a46 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:59.609831 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e37cbbe-fc51-44f5-b980-5347fa125a46 None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:59.610689 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e37cbbe-fc51-44f5-b980-5347fa125a46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:59.611195 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e37cbbe-fc51-44f5-b980-5347fa125a46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:59.612864 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e37cbbe-fc51-44f5-b980-5347fa125a46 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:59.627078 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 348/687] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:50:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:59.632523 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e37cbbe-fc51-44f5-b980-5347fa125a46 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:59.667534 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b643571-aa24-4b7b-93ff-2183bf3af9d9 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:59.691377 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:59.716698 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82b872c-766f-41bb-b675-9e32f450285a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:59.718246 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82b872c-766f-41bb-b675-9e32f450285a None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:59.718758 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82b872c-766f-41bb-b675-9e32f450285a None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:59.719234 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82b872c-766f-41bb-b675-9e32f450285a None admin] PATH_INFO: `/v3/domains` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:59.720083 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82b872c-766f-41bb-b675-9e32f450285a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:50:59.720621 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82b872c-766f-41bb-b675-9e32f450285a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:59.722946 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82b872c-766f-41bb-b675-9e32f450285a None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:59.740523 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 340/688] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:50:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:50:59.747565 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82b872c-766f-41bb-b675-9e32f450285a None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:59.766358 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e37cbbe-fc51-44f5-b980-5347fa125a46 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:59.780907 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:59.807018 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ad2e80-4bf0-4d79-b0cc-38249e0133c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:50:59.807799 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ad2e80-4bf0-4d79-b0cc-38249e0133c6 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:50:59.808183 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ad2e80-4bf0-4d79-b0cc-38249e0133c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:50:59.808593 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ad2e80-4bf0-4d79-b0cc-38249e0133c6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:50:59.818443 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ad2e80-4bf0-4d79-b0cc-38249e0133c6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:50:59.820139 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ad2e80-4bf0-4d79-b0cc-38249e0133c6 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:50:59.836017 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-67ad2e80-4bf0-4d79-b0cc-38249e0133c6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 06 21:50:59.836017 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:50:59.836017 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:50:59.836017 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:50:59.836017 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:50:59.836017 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:50:59.836017 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:50:59.836017 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:50:59.836017 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:50:59.836017 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:50:59.836017 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:50:59.836017 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:50:59.836017 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 21:50:59.836017 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 21:50:59.836017 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 21:50:59.836017 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 21:50:59.837069 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:50:59.837069 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:50:59.837069 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 21:50:59.837069 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 21:50:59.837069 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 21:50:59.837069 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 21:50:59.837069 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:50:59.837069 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:50:59.837069 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:50:59.837069 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:50:59.837069 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:50:59.837069 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:50:59.837069 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:50:59.837069 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:50:59.837069 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:50:59.837069 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:50:59.837069 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:50:59.837069 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:50:59.838041 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:50:59.838041 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:50:59.838041 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 21:50:59.838041 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 21:50:59.838041 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 21:50:59.838041 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 21:50:59.838041 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 21:50:59.838041 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 21:50:59.838041 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 06 21:50:59.838041 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:50:59.843101 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 349/689] 10.222.0.18 () {56 vars in 1093 bytes} [Sat Aug 6 21:50:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:50:59.843344 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ad2e80-4bf0-4d79-b0cc-38249e0133c6 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:50:59.965577 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ad2e80-4bf0-4d79-b0cc-38249e0133c6 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:50:59.967937 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:50:59.971061 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82b872c-766f-41bb-b675-9e32f450285a None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:00.027518 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b36567-15c8-460c-90f0-04a3e19ca965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:00.028133 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b36567-15c8-460c-90f0-04a3e19ca965 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:00.028450 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b36567-15c8-460c-90f0-04a3e19ca965 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:00.028730 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b36567-15c8-460c-90f0-04a3e19ca965 None admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:00.029465 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b36567-15c8-460c-90f0-04a3e19ca965 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:00.029768 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b36567-15c8-460c-90f0-04a3e19ca965 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:00.031344 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b36567-15c8-460c-90f0-04a3e19ca965 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:00.051929 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5684b95-b6c7-40ea-8110-ad4387d37f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:00.052591 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5684b95-b6c7-40ea-8110-ad4387d37f33 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:00.052876 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5684b95-b6c7-40ea-8110-ad4387d37f33 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:00.053151 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5684b95-b6c7-40ea-8110-ad4387d37f33 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:00.060778 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 350/690] 10.222.0.18 () {56 vars in 1133 bytes} [Sat Aug 6 21:50:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:51:00.070514 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5684b95-b6c7-40ea-8110-ad4387d37f33 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:00.071988 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5684b95-b6c7-40ea-8110-ad4387d37f33 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:00.073382 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b36567-15c8-460c-90f0-04a3e19ca965 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:00.088811 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-c5684b95-b6c7-40ea-8110-ad4387d37f33 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 06 21:51:00.088811 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:00.088811 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:00.088811 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:00.088811 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:00.088811 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:00.088811 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:00.088811 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:00.088811 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:00.088811 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:00.088811 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:00.088811 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:00.088811 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 21:51:00.088811 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 21:51:00.088811 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 21:51:00.088811 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 21:51:00.089794 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:00.089794 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:00.089794 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 21:51:00.089794 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 21:51:00.089794 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 21:51:00.089794 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 21:51:00.089794 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:00.089794 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:00.089794 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:00.089794 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:00.089794 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:00.089794 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:00.089794 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:00.089794 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:00.089794 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:00.089794 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:00.089794 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:00.089794 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:00.090798 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:00.090798 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:00.090798 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 21:51:00.090798 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 21:51:00.090798 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 21:51:00.090798 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 21:51:00.090798 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 21:51:00.090798 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 21:51:00.090798 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 06 21:51:00.090798 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:00.090798 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 341/691] 10.222.0.18 () {56 vars in 1096 bytes} [Sat Aug 6 21:50:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:00.096056 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5684b95-b6c7-40ea-8110-ad4387d37f33 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:00.240851 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:00.252435 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b36567-15c8-460c-90f0-04a3e19ca965 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:00.276837 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5684b95-b6c7-40ea-8110-ad4387d37f33 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:00.293287 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070cc964-6fc7-43af-b710-bd764e0828e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:00.293980 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070cc964-6fc7-43af-b710-bd764e0828e8 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:00.294924 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070cc964-6fc7-43af-b710-bd764e0828e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:00.295278 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070cc964-6fc7-43af-b710-bd764e0828e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:00.342566 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:00.343940 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837064e8-47d8-4237-b160-294290364670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:00.344567 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837064e8-47d8-4237-b160-294290364670 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:00.344842 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837064e8-47d8-4237-b160-294290364670 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:00.345109 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837064e8-47d8-4237-b160-294290364670 None admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:00.345840 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837064e8-47d8-4237-b160-294290364670 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:00.346660 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837064e8-47d8-4237-b160-294290364670 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:00.348957 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837064e8-47d8-4237-b160-294290364670 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:00.390191 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 342/692] 10.222.0.18 () {56 vars in 1135 bytes} [Sat Aug 6 21:51:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:51:00.391262 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837064e8-47d8-4237-b160-294290364670 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:00.420550 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070cc964-6fc7-43af-b710-bd764e0828e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:00.422759 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070cc964-6fc7-43af-b710-bd764e0828e8 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:00.529787 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837064e8-47d8-4237-b160-294290364670 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:00.565720 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed38476-8a45-45fd-8662-a8670e253a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:00.566324 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed38476-8a45-45fd-8662-a8670e253a3d None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:00.566599 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed38476-8a45-45fd-8662-a8670e253a3d None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:00.566870 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed38476-8a45-45fd-8662-a8670e253a3d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:00.572564 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 351/693] 10.222.0.18 () {58 vars in 1282 bytes} [Sat Aug 6 21:51:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 21:51:00.573919 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb704ba-8c49-426d-9ee7-4c57b2a9d9ee None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:00.574194 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb704ba-8c49-426d-9ee7-4c57b2a9d9ee None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:00.574460 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb704ba-8c49-426d-9ee7-4c57b2a9d9ee None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:00.575828 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 352/694] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:51:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:51:00.576457 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb704ba-8c49-426d-9ee7-4c57b2a9d9ee None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:00.592336 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:00.648317 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:00.660368 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed38476-8a45-45fd-8662-a8670e253a3d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:00.662017 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed38476-8a45-45fd-8662-a8670e253a3d None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:00.699834 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb704ba-8c49-426d-9ee7-4c57b2a9d9ee None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:00.745722 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae92169d-1846-4c42-b7cb-f3034e831125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:00.746729 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae92169d-1846-4c42-b7cb-f3034e831125 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:00.747108 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae92169d-1846-4c42-b7cb-f3034e831125 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:00.747458 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae92169d-1846-4c42-b7cb-f3034e831125 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:00.766297 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae92169d-1846-4c42-b7cb-f3034e831125 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:00.767860 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae92169d-1846-4c42-b7cb-f3034e831125 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:00.788975 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-ae92169d-1846-4c42-b7cb-f3034e831125 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:51:00.788975 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:00.788975 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:00.788975 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:00.788975 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:00.788975 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:00.788975 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:00.788975 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:00.788975 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:00.788975 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:00.788975 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:00.788975 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:00.788975 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:51:00.788975 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:51:00.788975 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:51:00.788975 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:51:00.789950 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:00.789950 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:00.789950 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:00.789950 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:00.789950 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:00.789950 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:00.789950 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:00.789950 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:00.789950 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:00.789950 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:00.789950 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:00.789950 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:00.789950 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:00.789950 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:00.789950 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:00.789950 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:00.789950 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:51:00.790887 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:51:00.790887 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:51:00.790887 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:00.791733 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 343/696] 10.222.0.18 () {58 vars in 1282 bytes} [Sat Aug 6 21:51:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 21:51:00.793163 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 353/696] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:51:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:00.793515 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a02f403-433c-4369-a504-aaaed7041bb1 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:00.793938 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a02f403-433c-4369-a504-aaaed7041bb1 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:00.794060 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae92169d-1846-4c42-b7cb-f3034e831125 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:00.794451 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a02f403-433c-4369-a504-aaaed7041bb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:00.817602 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-3a02f403-433c-4369-a504-aaaed7041bb1 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:51:00.819029 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-3a02f403-433c-4369-a504-aaaed7041bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:51:00.859454 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:00.900832 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 344/697] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:51:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 21:51:00.901890 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a02f403-433c-4369-a504-aaaed7041bb1 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:00.909546 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae92169d-1846-4c42-b7cb-f3034e831125 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:00.932258 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:00.965919 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292e248f-3dd2-4a1a-9afa-5c92d6b606f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:00.967762 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292e248f-3dd2-4a1a-9afa-5c92d6b606f2 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:00.968357 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292e248f-3dd2-4a1a-9afa-5c92d6b606f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:00.969273 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292e248f-3dd2-4a1a-9afa-5c92d6b606f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:00.985443 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292e248f-3dd2-4a1a-9afa-5c92d6b606f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:00.987941 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292e248f-3dd2-4a1a-9afa-5c92d6b606f2 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:01.014233 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-292e248f-3dd2-4a1a-9afa-5c92d6b606f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:51:01.014233 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:01.014233 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:01.014233 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:01.014233 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:01.014233 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:01.014233 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:01.014233 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:01.014233 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:01.014233 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:01.014233 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:01.014233 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:01.014233 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:51:01.014233 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:51:01.014233 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:51:01.014233 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:51:01.016457 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:01.016457 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:01.016457 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:01.016457 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:01.016457 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:01.016457 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:01.016457 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:01.016457 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:01.016457 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:01.016457 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:01.016457 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:01.016457 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:01.016457 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:01.016457 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:01.016457 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:01.016457 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:01.016457 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:51:01.017361 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:51:01.017361 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:51:01.017361 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:01.018088 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 354/698] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:51:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:01.018877 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292e248f-3dd2-4a1a-9afa-5c92d6b606f2 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:01.029261 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a02f403-433c-4369-a504-aaaed7041bb1 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:01.068944 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0acc0af-2e5c-445c-b1fc-d39b314681d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:01.069500 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0acc0af-2e5c-445c-b1fc-d39b314681d7 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:01.069779 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0acc0af-2e5c-445c-b1fc-d39b314681d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:01.070052 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0acc0af-2e5c-445c-b1fc-d39b314681d7 None admin] PATH_INFO: `/v3/domains` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:01.070697 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0acc0af-2e5c-445c-b1fc-d39b314681d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:01.070995 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0acc0af-2e5c-445c-b1fc-d39b314681d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:01.072526 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0acc0af-2e5c-445c-b1fc-d39b314681d7 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:01.089840 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 345/699] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:51:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:01.089840 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0acc0af-2e5c-445c-b1fc-d39b314681d7 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:01.122643 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:01.142351 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292e248f-3dd2-4a1a-9afa-5c92d6b606f2 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:01.186976 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34c4247-fd46-4c0d-8950-8c2fa5a736f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:01.188594 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c4247-fd46-4c0d-8950-8c2fa5a736f2 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:01.189133 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c4247-fd46-4c0d-8950-8c2fa5a736f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:01.189268 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c4247-fd46-4c0d-8950-8c2fa5a736f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:01.198055 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0acc0af-2e5c-445c-b1fc-d39b314681d7 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:01.199945 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34c4247-fd46-4c0d-8950-8c2fa5a736f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:01.201443 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34c4247-fd46-4c0d-8950-8c2fa5a736f2 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:01.215849 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-b34c4247-fd46-4c0d-8950-8c2fa5a736f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:51:01.215849 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:01.215849 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:01.215849 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:01.215849 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:01.215849 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:01.215849 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:01.215849 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:01.215849 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:01.215849 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:01.215849 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:01.215849 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:01.215849 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:51:01.215849 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:51:01.215849 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:51:01.215849 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:51:01.217040 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:01.217040 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:01.217040 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:01.217040 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:01.217040 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:01.217040 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:01.217040 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:01.217040 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:01.217040 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:01.217040 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:01.217040 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:01.217040 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:01.217040 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:01.217040 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:01.217040 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:01.217040 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:01.217040 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:51:01.218138 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:51:01.218138 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:51:01.218138 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:01.220542 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 355/700] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:51:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:01.221492 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34c4247-fd46-4c0d-8950-8c2fa5a736f2 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:01.237856 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65954e3-7e20-4543-b010-b7ee11bde04a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:01.238580 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65954e3-7e20-4543-b010-b7ee11bde04a None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:01.238950 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65954e3-7e20-4543-b010-b7ee11bde04a None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:01.239319 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65954e3-7e20-4543-b010-b7ee11bde04a None admin] PATH_INFO: `/v3/domains` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:01.240055 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65954e3-7e20-4543-b010-b7ee11bde04a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:01.240476 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65954e3-7e20-4543-b010-b7ee11bde04a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:01.242041 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65954e3-7e20-4543-b010-b7ee11bde04a None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:01.245437 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:01.259022 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 346/701] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:51:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:01.259022 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9162d8f8-cbd2-47cf-b89d-e5916c0d9147 None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:01.259022 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9162d8f8-cbd2-47cf-b89d-e5916c0d9147 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:01.259022 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9162d8f8-cbd2-47cf-b89d-e5916c0d9147 None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:01.259022 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 347/702] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:51:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:51:01.259022 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9162d8f8-cbd2-47cf-b89d-e5916c0d9147 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:01.343385 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34c4247-fd46-4c0d-8950-8c2fa5a736f2 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:01.367405 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:01.378874 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9162d8f8-cbd2-47cf-b89d-e5916c0d9147 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:01.388088 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6ae639-eb24-4ab2-b8ae-8d7f29f2d468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:01.388754 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6ae639-eb24-4ab2-b8ae-8d7f29f2d468 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:01.389085 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6ae639-eb24-4ab2-b8ae-8d7f29f2d468 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:01.389572 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6ae639-eb24-4ab2-b8ae-8d7f29f2d468 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:01.396711 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6ae639-eb24-4ab2-b8ae-8d7f29f2d468 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:01.398154 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6ae639-eb24-4ab2-b8ae-8d7f29f2d468 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:01.416216 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d019c4-a1be-4385-b321-c902539f1215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:01.416869 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d019c4-a1be-4385-b321-c902539f1215 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:01.417148 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d019c4-a1be-4385-b321-c902539f1215 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:01.417418 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d019c4-a1be-4385-b321-c902539f1215 None admin] PATH_INFO: `/v3/domains` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:01.418084 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d019c4-a1be-4385-b321-c902539f1215 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:01.418388 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d019c4-a1be-4385-b321-c902539f1215 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:01.419807 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d019c4-a1be-4385-b321-c902539f1215 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:01.438222 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 348/703] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:51:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:01.438222 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d019c4-a1be-4385-b321-c902539f1215 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:01.526707 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-3f6ae639-eb24-4ab2-b8ae-8d7f29f2d468 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 21:51:01.526707 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:01.526707 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:01.526707 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:01.526707 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:01.526707 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:01.526707 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:01.526707 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:01.526707 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:01.526707 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:01.526707 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:01.526707 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:01.526707 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 21:51:01.526707 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 21:51:01.526707 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 21:51:01.527689 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 21:51:01.527689 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:01.527689 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:01.527689 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:01.527689 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:01.527689 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:01.527689 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:01.527689 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:01.527689 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:01.527689 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:01.527689 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:01.527689 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:01.527689 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:01.527689 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:01.527689 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:01.527689 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:01.527689 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:01.528601 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 21:51:01.528601 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 21:51:01.528601 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 21:51:01.528601 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 21:51:01.528601 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 21:51:01.528601 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 21:51:01.528601 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 21:51:01.528601 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:01.528601 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 356/704] 10.222.0.18 () {56 vars in 1111 bytes} [Sat Aug 6 21:51:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:01.529115 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382a6b6d-73b8-447e-9d77-ed1eb5b536fe None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:01.529215 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382a6b6d-73b8-447e-9d77-ed1eb5b536fe None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:01.529818 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382a6b6d-73b8-447e-9d77-ed1eb5b536fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:01.548004 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-382a6b6d-73b8-447e-9d77-ed1eb5b536fe None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:51:01.549230 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-382a6b6d-73b8-447e-9d77-ed1eb5b536fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:51:01.554164 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d019c4-a1be-4385-b321-c902539f1215 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:01.590732 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c3333b-a109-4c57-ab75-9997d1bca462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:01.592160 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c3333b-a109-4c57-ab75-9997d1bca462 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:01.592674 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c3333b-a109-4c57-ab75-9997d1bca462 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:01.593147 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c3333b-a109-4c57-ab75-9997d1bca462 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:01.601539 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c3333b-a109-4c57-ab75-9997d1bca462 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:01.603034 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c3333b-a109-4c57-ab75-9997d1bca462 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:01.614998 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-f1c3333b-a109-4c57-ab75-9997d1bca462 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 21:51:01.614998 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:01.614998 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:01.614998 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:01.614998 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:01.614998 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:01.614998 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:01.614998 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:01.614998 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:01.614998 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:01.614998 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:01.614998 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:01.614998 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 21:51:01.614998 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 21:51:01.614998 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 21:51:01.615897 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 21:51:01.615897 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:01.615897 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:01.615897 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:01.615897 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:01.615897 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:01.615897 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:01.615897 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:01.615897 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:01.615897 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:01.615897 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:01.615897 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:01.615897 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:01.615897 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:01.615897 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:01.615897 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:01.615897 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:01.616826 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 21:51:01.616826 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 21:51:01.616826 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 21:51:01.616826 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 21:51:01.616826 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 21:51:01.616826 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 21:51:01.616826 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 21:51:01.616826 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:01.616826 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 349/705] 10.222.0.18 () {56 vars in 1111 bytes} [Sat Aug 6 21:51:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:01.616826 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c3333b-a109-4c57-ab75-9997d1bca462 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:01.636575 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 357/706] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:51:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 21:51:01.636575 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382a6b6d-73b8-447e-9d77-ed1eb5b536fe None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:01.642737 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:01.646678 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:01.737933 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c3333b-a109-4c57-ab75-9997d1bca462 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:01.738317 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382a6b6d-73b8-447e-9d77-ed1eb5b536fe None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:01.771977 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6521fa7e-05e7-4a97-85b4-65d32a38e244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:01.772635 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6521fa7e-05e7-4a97-85b4-65d32a38e244 None admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:01.772930 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6521fa7e-05e7-4a97-85b4-65d32a38e244 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:01.773172 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6521fa7e-05e7-4a97-85b4-65d32a38e244 None admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:01.773834 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6521fa7e-05e7-4a97-85b4-65d32a38e244 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:01.775250 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6521fa7e-05e7-4a97-85b4-65d32a38e244 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:01.776041 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51acd95-84f0-4631-8b84-a7126d48a1f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:01.776872 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51acd95-84f0-4631-8b84-a7126d48a1f1 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:01.777326 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51acd95-84f0-4631-8b84-a7126d48a1f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:01.777859 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51acd95-84f0-4631-8b84-a7126d48a1f1 None admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:01.778720 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51acd95-84f0-4631-8b84-a7126d48a1f1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:01.779220 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51acd95-84f0-4631-8b84-a7126d48a1f1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:01.780829 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51acd95-84f0-4631-8b84-a7126d48a1f1 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:01.793491 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 358/707] 10.222.0.18 () {56 vars in 1148 bytes} [Sat Aug 6 21:51:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:01.794295 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51acd95-84f0-4631-8b84-a7126d48a1f1 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:01.862533 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:01.865277 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 350/708] 10.222.0.18 () {60 vars in 1132 bytes} [Sat Aug 6 21:51:01 2022] POST /identity/v3/users => generated 254 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 21:51:01.870249 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6521fa7e-05e7-4a97-85b4-65d32a38e244 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:01.912220 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51acd95-84f0-4631-8b84-a7126d48a1f1 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:01.952687 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:01.953405 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86f6da7-f39d-4244-8645-c4439a37859b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:01.954376 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86f6da7-f39d-4244-8645-c4439a37859b None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:01.954376 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86f6da7-f39d-4244-8645-c4439a37859b None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:01.954669 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86f6da7-f39d-4244-8645-c4439a37859b None admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:01.955371 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86f6da7-f39d-4244-8645-c4439a37859b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:01.955678 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86f6da7-f39d-4244-8645-c4439a37859b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:01.957202 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86f6da7-f39d-4244-8645-c4439a37859b None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:01.976431 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 359/709] 10.222.0.18 () {56 vars in 1148 bytes} [Sat Aug 6 21:51:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:01.976431 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86f6da7-f39d-4244-8645-c4439a37859b None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:01.998329 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6521fa7e-05e7-4a97-85b4-65d32a38e244 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:02.039068 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c9a01a-9941-44b0-9197-df6255a98a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:02.040656 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c9a01a-9941-44b0-9197-df6255a98a64 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:02.040961 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c9a01a-9941-44b0-9197-df6255a98a64 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:02.041230 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c9a01a-9941-44b0-9197-df6255a98a64 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:02.041836 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c9a01a-9941-44b0-9197-df6255a98a64 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:02.043204 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c9a01a-9941-44b0-9197-df6255a98a64 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:02.056733 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-b0c9a01a-9941-44b0-9197-df6255a98a64 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 06 21:51:02.056733 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:02.056733 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 21:51:02.056733 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 21:51:02.056733 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 21:51:02.056733 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 21:51:02.056733 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 21:51:02.056733 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 21:51:02.056733 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 06 21:51:02.056733 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:02.056733 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 21:51:02.056733 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:02.056733 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:02.056733 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:02.056733 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:02.056733 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:02.056733 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:02.057843 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:02.057843 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:02.057843 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:02.057843 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:02.057843 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:02.057843 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:02.057843 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 21:51:02.057843 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 21:51:02.057843 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 21:51:02.057843 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 21:51:02.057843 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:02.057843 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:02.057843 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:02.057843 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:02.057843 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:02.057843 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:02.058880 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:02.058880 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:02.058880 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:02.058880 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:02.058880 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:02.058880 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:02.058880 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:02.058880 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:02.058880 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:02.058880 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:02.058880 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 21:51:02.058880 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 21:51:02.058880 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 06 21:51:02.058880 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:02.058880 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 351/710] 10.222.0.18 () {56 vars in 1129 bytes} [Sat Aug 6 21:51:01 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:02.059892 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d22015-b17d-45da-87d5-01b766aa106b None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:02.059892 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d22015-b17d-45da-87d5-01b766aa106b None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:02.060048 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d22015-b17d-45da-87d5-01b766aa106b None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:02.061356 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 352/711] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:51:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:51:02.066420 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d22015-b17d-45da-87d5-01b766aa106b None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:02.096541 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86f6da7-f39d-4244-8645-c4439a37859b None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:02.126714 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:02.135260 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5dcf23-f751-4d25-ad4e-5e2fde5fc247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:02.138476 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5dcf23-f751-4d25-ad4e-5e2fde5fc247 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:02.138476 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5dcf23-f751-4d25-ad4e-5e2fde5fc247 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:02.138476 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5dcf23-f751-4d25-ad4e-5e2fde5fc247 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:02.138476 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5dcf23-f751-4d25-ad4e-5e2fde5fc247 None admin] RBAC: Adding query filter params (role.id=4b8dcf0491fd4072acc69fd894bf77d0, scope.project.id=4bb6307eb0644d85a1d1daaa61352697, user.id=2a1319d33e9144859c6010843be312a3) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:02.138476 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5dcf23-f751-4d25-ad4e-5e2fde5fc247 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:02.139415 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5dcf23-f751-4d25-ad4e-5e2fde5fc247 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:02.152481 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 360/712] 10.222.0.18 () {56 vars in 1374 bytes} [Sat Aug 6 21:51:01 2022] GET /identity/v3/role_assignments?user.id=2a1319d33e9144859c6010843be312a3&scope.project.id=4bb6307eb0644d85a1d1daaa61352697&role.id=4b8dcf0491fd4072acc69fd894bf77d0 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:02.152481 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5dcf23-f751-4d25-ad4e-5e2fde5fc247 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:02.202724 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d22015-b17d-45da-87d5-01b766aa106b None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:02.238466 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:02.245622 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f993cce-702a-4705-bb5a-04340aaea7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:02.246804 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f993cce-702a-4705-bb5a-04340aaea7c3 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:02.246804 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f993cce-702a-4705-bb5a-04340aaea7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:02.247100 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f993cce-702a-4705-bb5a-04340aaea7c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:02.247893 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f993cce-702a-4705-bb5a-04340aaea7c3 None admin] RBAC: Adding query filter params (role.id=d90649d4ef604eeb8edcf0aa22932ca1, scope.project.id=4bb6307eb0644d85a1d1daaa61352697, user.id=e7fc9da7505744608dfd3bbd46a0a9d8) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:02.248261 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f993cce-702a-4705-bb5a-04340aaea7c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:02.250727 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f993cce-702a-4705-bb5a-04340aaea7c3 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:02.264548 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 353/713] 10.222.0.18 () {56 vars in 1374 bytes} [Sat Aug 6 21:51:02 2022] GET /identity/v3/role_assignments?user.id=e7fc9da7505744608dfd3bbd46a0a9d8&scope.project.id=4bb6307eb0644d85a1d1daaa61352697&role.id=d90649d4ef604eeb8edcf0aa22932ca1 => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:02.266486 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22806a18-d8d5-4133-8015-7dc72f030fc6 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:02.266871 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22806a18-d8d5-4133-8015-7dc72f030fc6 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:02.267238 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22806a18-d8d5-4133-8015-7dc72f030fc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:02.292675 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5dcf23-f751-4d25-ad4e-5e2fde5fc247 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:02.305640 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-22806a18-d8d5-4133-8015-7dc72f030fc6 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:51:02.307054 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-22806a18-d8d5-4133-8015-7dc72f030fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:51:02.339217 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5459befa-015e-46b9-9708-fbc1d6e0ddcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:02.340091 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5459befa-015e-46b9-9708-fbc1d6e0ddcf None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:02.340572 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5459befa-015e-46b9-9708-fbc1d6e0ddcf None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:02.341005 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5459befa-015e-46b9-9708-fbc1d6e0ddcf None admin] PATH_INFO: `/v3/services` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:02.341794 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5459befa-015e-46b9-9708-fbc1d6e0ddcf None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:02.342786 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5459befa-015e-46b9-9708-fbc1d6e0ddcf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:02.345761 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5459befa-015e-46b9-9708-fbc1d6e0ddcf None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:02.352797 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 361/714] 10.222.0.18 () {56 vars in 1124 bytes} [Sat Aug 6 21:51:02 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:02.359481 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5459befa-015e-46b9-9708-fbc1d6e0ddcf None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:02.424515 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 354/715] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:51:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 21:51:02.441236 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22806a18-d8d5-4133-8015-7dc72f030fc6 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:02.444872 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:02.507861 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5459befa-015e-46b9-9708-fbc1d6e0ddcf None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:02.547877 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfe03c6-cceb-413a-a000-de6c9ffb164c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:02.548530 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfe03c6-cceb-413a-a000-de6c9ffb164c None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:02.548989 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfe03c6-cceb-413a-a000-de6c9ffb164c None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:02.549262 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfe03c6-cceb-413a-a000-de6c9ffb164c None admin] PATH_INFO: `/v3/services` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:02.549897 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfe03c6-cceb-413a-a000-de6c9ffb164c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:02.550332 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfe03c6-cceb-413a-a000-de6c9ffb164c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:02.551533 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfe03c6-cceb-413a-a000-de6c9ffb164c None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:02.557665 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 362/716] 10.222.0.18 () {56 vars in 1124 bytes} [Sat Aug 6 21:51:02 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:02.600504 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22806a18-d8d5-4133-8015-7dc72f030fc6 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:02.641424 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:02.650443 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f609b8cd-d60d-41c1-b42c-1375b93ae061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:02.651275 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f609b8cd-d60d-41c1-b42c-1375b93ae061 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:02.651648 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f609b8cd-d60d-41c1-b42c-1375b93ae061 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:02.652016 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f609b8cd-d60d-41c1-b42c-1375b93ae061 None admin] PATH_INFO: `/v3/services/network` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:02.652786 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f609b8cd-d60d-41c1-b42c-1375b93ae061 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:02.654352 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f609b8cd-d60d-41c1-b42c-1375b93ae061 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:02.665386 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-f609b8cd-d60d-41c1-b42c-1375b93ae061 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 06 21:51:02.665386 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:02.665386 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 21:51:02.665386 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 21:51:02.665386 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 21:51:02.665386 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 21:51:02.665386 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 21:51:02.665386 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 21:51:02.665386 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 06 21:51:02.665386 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:02.665386 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 21:51:02.665386 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:02.665386 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:02.665386 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:02.665386 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:02.665386 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:02.665386 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:02.666637 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:02.666637 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:02.666637 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:02.666637 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:02.666637 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:02.666637 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:02.666637 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 21:51:02.666637 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 21:51:02.666637 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 21:51:02.666637 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 21:51:02.666637 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:02.666637 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:02.666637 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:02.666637 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:02.666637 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:02.666637 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:02.667668 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:02.667668 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:02.667668 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:02.667668 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:02.667668 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:02.667668 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:02.667668 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:02.667668 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:02.667668 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:02.667668 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:02.667668 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 21:51:02.667668 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 21:51:02.667668 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 06 21:51:02.667668 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:02.669433 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 355/717] 10.222.0.18 () {56 vars in 1111 bytes} [Sat Aug 6 21:51:02 2022] GET /identity/v3/services/network => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 21:51:02.676318 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfe03c6-cceb-413a-a000-de6c9ffb164c None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:02.802244 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfe03c6-cceb-413a-a000-de6c9ffb164c None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:02.824649 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:02.838917 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4647772-d3da-44aa-a415-3275f7e0a93f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:02.839581 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4647772-d3da-44aa-a415-3275f7e0a93f None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:02.839865 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4647772-d3da-44aa-a415-3275f7e0a93f None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:02.840136 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4647772-d3da-44aa-a415-3275f7e0a93f None admin] PATH_INFO: `/v3/services` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:02.840846 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4647772-d3da-44aa-a415-3275f7e0a93f None admin] RBAC: Adding query filter params (name=network) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:02.841160 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4647772-d3da-44aa-a415-3275f7e0a93f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:02.842527 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4647772-d3da-44aa-a415-3275f7e0a93f None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:02.847865 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 363/718] 10.222.0.18 () {56 vars in 1112 bytes} [Sat Aug 6 21:51:02 2022] GET /identity/v3/services?name=network => generated 124 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:02.853383 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f609b8cd-d60d-41c1-b42c-1375b93ae061 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:02.978000 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f609b8cd-d60d-41c1-b42c-1375b93ae061 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:03.027955 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ee21ce-cbaa-426c-bf00-c1a3d67d0c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:03.028687 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee21ce-cbaa-426c-bf00-c1a3d67d0c0c None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:03.029032 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee21ce-cbaa-426c-bf00-c1a3d67d0c0c None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:03.029374 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee21ce-cbaa-426c-bf00-c1a3d67d0c0c None admin] PATH_INFO: `/v3/services` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:03.030082 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ee21ce-cbaa-426c-bf00-c1a3d67d0c0c None admin] RBAC: Adding query filter params (type=network) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:03.030452 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ee21ce-cbaa-426c-bf00-c1a3d67d0c0c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:03.032930 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ee21ce-cbaa-426c-bf00-c1a3d67d0c0c None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:03.045250 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 356/719] 10.222.0.18 () {56 vars in 1112 bytes} [Sat Aug 6 21:51:02 2022] GET /identity/v3/services?type=network => generated 124 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:51:04.108067 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181df974-d607-4ac4-a0af-ab1f15ce80d2 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:04.108616 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181df974-d607-4ac4-a0af-ab1f15ce80d2 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:04.108616 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181df974-d607-4ac4-a0af-ab1f15ce80d2 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:04.111226 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 364/720] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:51:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:51:04.116334 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb360ac-e502-4ae6-b8b3-835baecddc9e None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:04.116800 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb360ac-e502-4ae6-b8b3-835baecddc9e None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:04.116907 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb360ac-e502-4ae6-b8b3-835baecddc9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:04.141606 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-ecb360ac-e502-4ae6-b8b3-835baecddc9e None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:51:04.142818 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-ecb360ac-e502-4ae6-b8b3-835baecddc9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:51:04.204004 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:04.227803 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 357/721] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:51:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 21:51:04.273879 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181df974-d607-4ac4-a0af-ab1f15ce80d2 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:04.449168 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181df974-d607-4ac4-a0af-ab1f15ce80d2 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:04.495080 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fd3c18-2d21-4b13-9701-5edb74620456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:04.495711 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd3c18-2d21-4b13-9701-5edb74620456 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:04.495991 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd3c18-2d21-4b13-9701-5edb74620456 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:04.496266 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd3c18-2d21-4b13-9701-5edb74620456 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:04.508111 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fd3c18-2d21-4b13-9701-5edb74620456 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:51:04.512823 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-02fd3c18-2d21-4b13-9701-5edb74620456 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 21:51:04.512823 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:04.512823 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:04.512823 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:04.512823 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:04.512823 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:04.512823 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:04.512823 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:04.512823 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:04.512823 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:04.512823 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:04.512823 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:04.512823 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:51:04.512823 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:51:04.512823 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:51:04.512823 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:51:04.513833 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:51:04.513833 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:51:04.513833 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:51:04.513833 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:51:04.513833 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:04.513833 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:04.513833 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:04.513833 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:04.513833 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:04.513833 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:04.513833 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:04.513833 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:04.513833 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:04.513833 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:04.513833 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:04.513833 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:04.514706 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:04.514706 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:04.514706 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:04.514706 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:04.514706 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:51:04.514706 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:51:04.514706 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:51:04.514706 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:51:04.514706 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:51:04.514706 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:51:04.514706 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 21:51:04.514706 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:04.514706 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 365/722] 10.222.0.18 () {56 vars in 1102 bytes} [Sat Aug 6 21:51:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:04.520733 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb360ac-e502-4ae6-b8b3-835baecddc9e None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:04.649469 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb360ac-e502-4ae6-b8b3-835baecddc9e None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:04.653348 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d15c7-1782-4773-9e33-a24bc50588b4 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:04.653632 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d15c7-1782-4773-9e33-a24bc50588b4 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:04.653904 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d15c7-1782-4773-9e33-a24bc50588b4 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:04.655239 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 366/723] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:51:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 21:51:04.661007 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dc1282-4204-4210-8343-1bbe7d0c227c None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:04.661288 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dc1282-4204-4210-8343-1bbe7d0c227c None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:04.661628 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dc1282-4204-4210-8343-1bbe7d0c227c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:04.680823 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-47dc1282-4204-4210-8343-1bbe7d0c227c None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:51:04.682181 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-47dc1282-4204-4210-8343-1bbe7d0c227c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:51:04.698441 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7e4113-1470-46a5-a0eb-32947eda4fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:04.699231 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e4113-1470-46a5-a0eb-32947eda4fb1 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:04.699459 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e4113-1470-46a5-a0eb-32947eda4fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:04.699740 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e4113-1470-46a5-a0eb-32947eda4fb1 None admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:04.700440 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7e4113-1470-46a5-a0eb-32947eda4fb1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:04.700755 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7e4113-1470-46a5-a0eb-32947eda4fb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:04.702260 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7e4113-1470-46a5-a0eb-32947eda4fb1 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:04.716956 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 358/724] 10.222.0.18 () {56 vars in 1103 bytes} [Sat Aug 6 21:51:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:51:04.718843 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7e4113-1470-46a5-a0eb-32947eda4fb1 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:04.751158 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:04.800236 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:04.804316 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 367/725] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:51:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 21:51:04.825485 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dc1282-4204-4210-8343-1bbe7d0c227c None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:04.934034 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dc1282-4204-4210-8343-1bbe7d0c227c None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:04.937348 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7e4113-1470-46a5-a0eb-32947eda4fb1 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:04.973214 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8ce0db-fcb0-48f6-8612-e7513892b06d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:04.973970 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8ce0db-fcb0-48f6-8612-e7513892b06d None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:04.974249 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8ce0db-fcb0-48f6-8612-e7513892b06d None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:04.974517 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8ce0db-fcb0-48f6-8612-e7513892b06d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:04.986937 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8ce0db-fcb0-48f6-8612-e7513892b06d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:51:04.992059 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-0b8ce0db-fcb0-48f6-8612-e7513892b06d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 21:51:04.992059 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:04.992059 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:04.992059 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:04.992059 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:04.992059 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:04.992059 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:04.992059 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:04.992059 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:04.992059 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:04.992059 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:04.992059 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:04.992059 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:51:04.992059 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:51:04.992059 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:51:04.992059 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:51:04.993106 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:51:04.993106 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:51:04.993106 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:51:04.993106 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:51:04.993106 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:04.993106 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:04.993106 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:04.993106 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:04.993106 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:04.993106 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:04.993106 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:04.993106 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:04.993106 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:04.993106 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:04.993106 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:04.993106 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:04.993953 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:04.993953 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:04.993953 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:04.993953 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:04.993953 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:51:04.993953 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:51:04.993953 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:51:04.993953 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:51:04.993953 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:51:04.993953 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:51:04.993953 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 21:51:04.993953 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:04.993953 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 368/726] 10.222.0.18 () {56 vars in 1096 bytes} [Sat Aug 6 21:51:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 21:51:04.999831 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8ce0db-fcb0-48f6-8612-e7513892b06d None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:05.023404 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d480c18-efbf-4b8a-884f-2248b2efdf69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:05.024279 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d480c18-efbf-4b8a-884f-2248b2efdf69 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:05.024584 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d480c18-efbf-4b8a-884f-2248b2efdf69 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:05.024854 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d480c18-efbf-4b8a-884f-2248b2efdf69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:05.047896 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d480c18-efbf-4b8a-884f-2248b2efdf69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:05.050368 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d480c18-efbf-4b8a-884f-2248b2efdf69 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:05.066712 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:05.076391 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-0d480c18-efbf-4b8a-884f-2248b2efdf69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:51:05.076391 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:05.076391 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:05.076391 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:05.076391 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:05.076391 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:05.076391 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:05.076391 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:05.076391 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:05.076391 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:05.076391 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:05.076391 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:05.076391 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:51:05.076391 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:51:05.076391 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:51:05.076391 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:51:05.082738 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:05.082738 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:05.082738 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:05.082738 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:05.082738 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:05.082738 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:05.082738 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:05.082738 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:05.082738 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:05.082738 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:05.082738 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:05.082738 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:05.082738 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:05.082738 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:05.082738 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:05.082738 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:05.082738 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:51:05.083661 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:51:05.083661 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:51:05.083661 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:05.083661 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 359/727] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:51:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 21:51:05.083661 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d480c18-efbf-4b8a-884f-2248b2efdf69 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:05.188308 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8ce0db-fcb0-48f6-8612-e7513892b06d None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:05.229988 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db97d68-fca5-4b9d-87b6-b9ac5aa6f2d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:05.230994 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db97d68-fca5-4b9d-87b6-b9ac5aa6f2d6 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:05.231273 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db97d68-fca5-4b9d-87b6-b9ac5aa6f2d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:05.231940 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db97d68-fca5-4b9d-87b6-b9ac5aa6f2d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:05.233174 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db97d68-fca5-4b9d-87b6-b9ac5aa6f2d6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:05.233483 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db97d68-fca5-4b9d-87b6-b9ac5aa6f2d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:05.235405 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db97d68-fca5-4b9d-87b6-b9ac5aa6f2d6 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:05.250262 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 369/728] 10.222.0.18 () {56 vars in 1099 bytes} [Sat Aug 6 21:51:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:05.251731 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d8049-7be6-4d67-9c37-23f20525758e None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:05.252015 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d8049-7be6-4d67-9c37-23f20525758e None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:05.252293 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d8049-7be6-4d67-9c37-23f20525758e None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:05.253616 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:05.254353 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 370/729] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:51:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:51:05.255985 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12d8049-7be6-4d67-9c37-23f20525758e None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:05.260027 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d480c18-efbf-4b8a-884f-2248b2efdf69 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:05.294012 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4448d2ba-1df3-4c7c-8a57-bff1f3a91cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:05.294859 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4448d2ba-1df3-4c7c-8a57-bff1f3a91cb6 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:05.295148 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4448d2ba-1df3-4c7c-8a57-bff1f3a91cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:05.295396 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4448d2ba-1df3-4c7c-8a57-bff1f3a91cb6 None admin] PATH_INFO: `/v3/domains` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:05.296035 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4448d2ba-1df3-4c7c-8a57-bff1f3a91cb6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:05.296336 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4448d2ba-1df3-4c7c-8a57-bff1f3a91cb6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:05.297804 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4448d2ba-1df3-4c7c-8a57-bff1f3a91cb6 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:05.313774 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 360/730] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:51:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:05.313774 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cfb91b-4762-4035-add4-7ab199b02046 None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:05.313774 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cfb91b-4762-4035-add4-7ab199b02046 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:05.313774 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cfb91b-4762-4035-add4-7ab199b02046 None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:05.313774 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 361/731] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:51:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:51:05.313774 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa1da9f-1433-45e0-9861-702668ab19e2 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:05.314261 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa1da9f-1433-45e0-9861-702668ab19e2 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:05.314691 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa1da9f-1433-45e0-9861-702668ab19e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:05.332680 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-0fa1da9f-1433-45e0-9861-702668ab19e2 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:51:05.333847 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-0fa1da9f-1433-45e0-9861-702668ab19e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:51:05.374202 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:05.381924 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12d8049-7be6-4d67-9c37-23f20525758e None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:05.413139 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 362/732] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:51:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 21:51:05.414310 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa1da9f-1433-45e0-9861-702668ab19e2 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:05.424572 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa58ab6-bffb-4ff4-a9da-a5fb4ee6d0bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:05.425425 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa58ab6-bffb-4ff4-a9da-a5fb4ee6d0bc None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:05.425878 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa58ab6-bffb-4ff4-a9da-a5fb4ee6d0bc None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:05.426323 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa58ab6-bffb-4ff4-a9da-a5fb4ee6d0bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:05.436730 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa58ab6-bffb-4ff4-a9da-a5fb4ee6d0bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:05.438977 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa58ab6-bffb-4ff4-a9da-a5fb4ee6d0bc None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:05.454974 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-2aa58ab6-bffb-4ff4-a9da-a5fb4ee6d0bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:51:05.454974 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:05.454974 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:05.454974 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:05.454974 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:05.454974 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:05.454974 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:05.454974 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:05.454974 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:05.454974 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:05.454974 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:05.454974 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:05.454974 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:51:05.454974 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:51:05.454974 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:51:05.454974 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:51:05.455971 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:05.455971 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:05.455971 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:05.455971 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:05.455971 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:05.455971 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:05.455971 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:05.455971 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:05.455971 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:05.455971 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:05.455971 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:05.455971 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:05.455971 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:05.455971 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:05.455971 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:05.455971 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:05.455971 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:51:05.456880 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:51:05.456880 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:51:05.456880 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:05.456880 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 371/733] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:51:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:05.457515 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e07da2-21e0-4a29-bce1-48bb8bee5092 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:05.457783 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e07da2-21e0-4a29-bce1-48bb8bee5092 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:05.458060 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e07da2-21e0-4a29-bce1-48bb8bee5092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:05.475322 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-45e07da2-21e0-4a29-bce1-48bb8bee5092 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:51:05.476498 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-45e07da2-21e0-4a29-bce1-48bb8bee5092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:51:05.513023 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:05.531415 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa1da9f-1433-45e0-9861-702668ab19e2 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:05.545372 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 372/734] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:51:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 21:51:05.546013 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e07da2-21e0-4a29-bce1-48bb8bee5092 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:05.576918 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d3f0f8-8eeb-4a2a-b4ff-708ed85d2c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:05.577558 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d3f0f8-8eeb-4a2a-b4ff-708ed85d2c4a None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:05.578355 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d3f0f8-8eeb-4a2a-b4ff-708ed85d2c4a None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:05.578624 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d3f0f8-8eeb-4a2a-b4ff-708ed85d2c4a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:05.586143 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d3f0f8-8eeb-4a2a-b4ff-708ed85d2c4a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:05.587588 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d3f0f8-8eeb-4a2a-b4ff-708ed85d2c4a None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:05.599860 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-51d3f0f8-8eeb-4a2a-b4ff-708ed85d2c4a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 06 21:51:05.599860 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:05.599860 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:05.599860 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:05.599860 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:05.599860 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:05.599860 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:05.599860 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:05.599860 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:05.599860 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:05.599860 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:05.599860 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:05.599860 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 21:51:05.599860 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 21:51:05.599860 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 21:51:05.599860 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 21:51:05.600917 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:05.600917 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:05.600917 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 21:51:05.600917 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 21:51:05.600917 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 21:51:05.600917 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 21:51:05.600917 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:05.600917 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:05.600917 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:05.600917 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:05.600917 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:05.600917 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:05.600917 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:05.600917 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:05.600917 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:05.600917 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:05.600917 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:05.600917 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:05.602105 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:05.602105 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:05.602105 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 21:51:05.602105 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 21:51:05.602105 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 21:51:05.602105 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 21:51:05.602105 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 21:51:05.602105 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 21:51:05.602105 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 06 21:51:05.602105 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:05.604558 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 363/735] 10.222.0.18 () {56 vars in 1117 bytes} [Sat Aug 6 21:51:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:05.606113 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d3f0f8-8eeb-4a2a-b4ff-708ed85d2c4a None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:05.639606 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:05.662351 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e07da2-21e0-4a29-bce1-48bb8bee5092 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:05.707075 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b02bc8f-fede-4186-b0e3-b41fdebab802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:05.707687 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b02bc8f-fede-4186-b0e3-b41fdebab802 None admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:05.707964 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b02bc8f-fede-4186-b0e3-b41fdebab802 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:05.708235 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b02bc8f-fede-4186-b0e3-b41fdebab802 None admin] PATH_INFO: `/v3/services` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:05.708904 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b02bc8f-fede-4186-b0e3-b41fdebab802 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:05.710323 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b02bc8f-fede-4186-b0e3-b41fdebab802 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:05.715107 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d3f0f8-8eeb-4a2a-b4ff-708ed85d2c4a None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:05.750882 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ca46a6-a46e-4798-b54e-75fd1caf50c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:05.751479 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca46a6-a46e-4798-b54e-75fd1caf50c1 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:05.751753 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca46a6-a46e-4798-b54e-75fd1caf50c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:05.752018 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca46a6-a46e-4798-b54e-75fd1caf50c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:05.752688 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ca46a6-a46e-4798-b54e-75fd1caf50c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:05.752987 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ca46a6-a46e-4798-b54e-75fd1caf50c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:05.754342 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ca46a6-a46e-4798-b54e-75fd1caf50c1 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:05.765739 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 364/736] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:51:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:05.766425 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ca46a6-a46e-4798-b54e-75fd1caf50c1 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:05.774793 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:05.824179 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 373/737] 10.222.0.18 () {60 vars in 1141 bytes} [Sat Aug 6 21:51:05 2022] POST /identity/v3/services => generated 251 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 21:51:05.825074 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b02bc8f-fede-4186-b0e3-b41fdebab802 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:05.871865 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ca46a6-a46e-4798-b54e-75fd1caf50c1 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:05.924428 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45464fb3-c171-4d59-b8e2-616fcb6e8640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:05.925083 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45464fb3-c171-4d59-b8e2-616fcb6e8640 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:05.925361 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45464fb3-c171-4d59-b8e2-616fcb6e8640 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:05.925640 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45464fb3-c171-4d59-b8e2-616fcb6e8640 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:05.939103 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45464fb3-c171-4d59-b8e2-616fcb6e8640 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:51:05.940472 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b02bc8f-fede-4186-b0e3-b41fdebab802 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:05.944233 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-45464fb3-c171-4d59-b8e2-616fcb6e8640 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 21:51:05.944233 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:05.944233 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:05.944233 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:05.944233 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:05.944233 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:05.944233 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:05.944233 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:05.944233 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:05.944233 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:05.944233 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:05.944233 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:05.944233 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:51:05.944233 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:51:05.944233 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:51:05.944233 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:51:05.945252 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:51:05.945252 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:51:05.945252 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:51:05.945252 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:51:05.945252 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:05.945252 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:05.945252 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:05.945252 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:05.945252 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:05.945252 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:05.945252 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:05.945252 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:05.945252 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:05.945252 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:05.945252 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:05.945252 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:05.946090 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:05.946090 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:05.946090 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:05.946090 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:05.946090 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:51:05.946090 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:51:05.946090 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:51:05.946090 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:51:05.946090 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:51:05.946090 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:51:05.946090 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 21:51:05.946090 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:05.946090 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 365/738] 10.222.0.18 () {56 vars in 1096 bytes} [Sat Aug 6 21:51:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:05.947325 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c849415c-77fc-4b54-8c28-9dd30dae0a1a None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:05.947325 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c849415c-77fc-4b54-8c28-9dd30dae0a1a None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:05.947559 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c849415c-77fc-4b54-8c28-9dd30dae0a1a None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:05.952518 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 366/739] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:51:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:51:05.952518 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c849415c-77fc-4b54-8c28-9dd30dae0a1a None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:05.981943 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc94d36d-5baf-4da6-b793-87db7dcf4c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:05.982861 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc94d36d-5baf-4da6-b793-87db7dcf4c48 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:05.983077 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc94d36d-5baf-4da6-b793-87db7dcf4c48 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:05.983347 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc94d36d-5baf-4da6-b793-87db7dcf4c48 None admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:05.984257 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc94d36d-5baf-4da6-b793-87db7dcf4c48 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:05.984593 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc94d36d-5baf-4da6-b793-87db7dcf4c48 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:05.986220 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc94d36d-5baf-4da6-b793-87db7dcf4c48 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:05.998058 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:06.009045 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 374/740] 10.222.0.18 () {56 vars in 1149 bytes} [Sat Aug 6 21:51:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:06.009858 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc94d36d-5baf-4da6-b793-87db7dcf4c48 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:06.025011 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:06.068648 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c849415c-77fc-4b54-8c28-9dd30dae0a1a None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:06.105026 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f441efee-646a-4082-9e86-284982c7121e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:06.105662 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f441efee-646a-4082-9e86-284982c7121e None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:06.106014 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f441efee-646a-4082-9e86-284982c7121e None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:06.106296 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f441efee-646a-4082-9e86-284982c7121e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:06.115497 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f441efee-646a-4082-9e86-284982c7121e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:06.116973 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f441efee-646a-4082-9e86-284982c7121e None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:06.126336 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc94d36d-5baf-4da6-b793-87db7dcf4c48 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:06.135191 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-f441efee-646a-4082-9e86-284982c7121e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 06 21:51:06.135191 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:06.135191 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:06.135191 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:06.135191 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:06.135191 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:06.135191 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:06.135191 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:06.135191 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:06.135191 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:06.135191 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:06.135191 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:06.135191 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 21:51:06.135191 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 21:51:06.135191 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 21:51:06.135191 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 21:51:06.138450 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:06.138450 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:06.138450 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 21:51:06.138450 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 21:51:06.138450 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 21:51:06.138450 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 21:51:06.138450 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:06.138450 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:06.138450 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:06.138450 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:06.138450 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:06.138450 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:06.138450 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:06.138450 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:06.138450 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:06.138450 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:06.138450 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:06.138450 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:06.139476 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:06.139476 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:06.139476 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 21:51:06.139476 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 21:51:06.139476 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 21:51:06.139476 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 21:51:06.139476 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 21:51:06.139476 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 21:51:06.139476 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 06 21:51:06.139476 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:06.139476 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 367/741] 10.222.0.18 () {56 vars in 1093 bytes} [Sat Aug 6 21:51:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:06.139476 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302a5f4a-43f9-45dd-92d2-e6feb8c34636 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:06.139476 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302a5f4a-43f9-45dd-92d2-e6feb8c34636 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:06.140216 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302a5f4a-43f9-45dd-92d2-e6feb8c34636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:06.157725 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-302a5f4a-43f9-45dd-92d2-e6feb8c34636 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:51:06.158893 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-302a5f4a-43f9-45dd-92d2-e6feb8c34636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:51:06.168071 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb17ebc-0b8b-4cac-a499-96708a46ac5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:06.168850 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb17ebc-0b8b-4cac-a499-96708a46ac5c None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:06.169214 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb17ebc-0b8b-4cac-a499-96708a46ac5c None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:06.169535 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb17ebc-0b8b-4cac-a499-96708a46ac5c None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:06.170292 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb17ebc-0b8b-4cac-a499-96708a46ac5c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:06.170656 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb17ebc-0b8b-4cac-a499-96708a46ac5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:06.172719 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb17ebc-0b8b-4cac-a499-96708a46ac5c None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:06.186227 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 375/742] 10.222.0.18 () {56 vars in 1099 bytes} [Sat Aug 6 21:51:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:06.186227 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb17ebc-0b8b-4cac-a499-96708a46ac5c None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:06.233345 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 368/743] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:51:06 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 21:51:06.234969 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302a5f4a-43f9-45dd-92d2-e6feb8c34636 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:06.262738 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:06.290099 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:06.307939 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb17ebc-0b8b-4cac-a499-96708a46ac5c None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:06.342026 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302a5f4a-43f9-45dd-92d2-e6feb8c34636 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:06.350831 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b160788-cb5f-4886-aca8-b1240d402ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:06.351773 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b160788-cb5f-4886-aca8-b1240d402ae2 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:06.352052 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b160788-cb5f-4886-aca8-b1240d402ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:06.352751 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b160788-cb5f-4886-aca8-b1240d402ae2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:06.381067 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e25fd6-f351-4a7e-b1a9-7daaee3e2e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:06.381668 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e25fd6-f351-4a7e-b1a9-7daaee3e2e0a None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:06.381943 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e25fd6-f351-4a7e-b1a9-7daaee3e2e0a None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:06.382214 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e25fd6-f351-4a7e-b1a9-7daaee3e2e0a None admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:06.382945 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e25fd6-f351-4a7e-b1a9-7daaee3e2e0a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:06.383246 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e25fd6-f351-4a7e-b1a9-7daaee3e2e0a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:06.384695 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e25fd6-f351-4a7e-b1a9-7daaee3e2e0a None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:06.409785 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 369/744] 10.222.0.18 () {56 vars in 1133 bytes} [Sat Aug 6 21:51:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:06.409785 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e25fd6-f351-4a7e-b1a9-7daaee3e2e0a None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:06.430374 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b160788-cb5f-4886-aca8-b1240d402ae2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:06.432427 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b160788-cb5f-4886-aca8-b1240d402ae2 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:06.491551 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:06.516973 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e25fd6-f351-4a7e-b1a9-7daaee3e2e0a None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:06.528364 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:06.539404 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 376/745] 10.222.0.18 () {58 vars in 1282 bytes} [Sat Aug 6 21:51:06 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 21:51:06.539404 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b160788-cb5f-4886-aca8-b1240d402ae2 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:06.560513 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d819f293-a48b-491e-a98b-072758251f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:06.561077 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d819f293-a48b-491e-a98b-072758251f2c None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:06.561367 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d819f293-a48b-491e-a98b-072758251f2c None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:06.561615 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d819f293-a48b-491e-a98b-072758251f2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:06.572635 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d819f293-a48b-491e-a98b-072758251f2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:06.574471 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d819f293-a48b-491e-a98b-072758251f2c None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:06.589244 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-d819f293-a48b-491e-a98b-072758251f2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:51:06.589244 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:06.589244 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:06.589244 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:06.589244 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:06.589244 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:06.589244 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:06.589244 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:06.589244 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:06.589244 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:06.589244 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:06.589244 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:06.589244 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:51:06.589244 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:51:06.589244 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:51:06.589244 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:51:06.590407 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:06.590407 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:06.590407 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:06.590407 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:06.590407 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:06.590407 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:06.590407 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:06.590407 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:06.590407 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:06.590407 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:06.590407 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:06.590407 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:06.590407 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:06.590407 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:06.590407 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:06.590407 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:06.590407 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:51:06.591389 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:51:06.591389 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:51:06.591389 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:06.591389 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 370/746] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:51:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:06.591389 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d819f293-a48b-491e-a98b-072758251f2c None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:06.655282 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b160788-cb5f-4886-aca8-b1240d402ae2 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:06.694626 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8133574-1ee3-4378-8566-4997e93380c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:06.695449 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8133574-1ee3-4378-8566-4997e93380c2 None admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:06.695880 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8133574-1ee3-4378-8566-4997e93380c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:06.696332 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8133574-1ee3-4378-8566-4997e93380c2 None admin] PATH_INFO: `/v3/services` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:06.697198 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8133574-1ee3-4378-8566-4997e93380c2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:06.698715 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8133574-1ee3-4378-8566-4997e93380c2 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:06.704384 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d819f293-a48b-491e-a98b-072758251f2c None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:06.738586 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26974d4-3261-4e77-83c7-eca594cbcd06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:06.739222 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26974d4-3261-4e77-83c7-eca594cbcd06 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:06.739503 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26974d4-3261-4e77-83c7-eca594cbcd06 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:06.739778 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26974d4-3261-4e77-83c7-eca594cbcd06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:06.750536 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26974d4-3261-4e77-83c7-eca594cbcd06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:06.752001 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26974d4-3261-4e77-83c7-eca594cbcd06 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:06.766320 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-d26974d4-3261-4e77-83c7-eca594cbcd06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:51:06.766320 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:06.766320 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:06.766320 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:06.766320 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:06.766320 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:06.766320 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:06.766320 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:06.766320 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:06.766320 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:06.766320 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:06.766320 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:06.766320 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:51:06.766320 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:51:06.766320 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:51:06.766320 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:51:06.767430 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:06.767430 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:06.767430 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:06.767430 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:06.767430 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:06.767430 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:06.767430 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:06.767430 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:06.767430 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:06.767430 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:06.767430 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:06.767430 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:06.767430 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:06.767430 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:06.767430 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:06.767430 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:06.767430 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:51:06.768344 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:51:06.768344 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:51:06.768344 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:06.768537 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 377/747] 10.222.0.18 () {60 vars in 1141 bytes} [Sat Aug 6 21:51:06 2022] POST /identity/v3/services => generated 240 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 21:51:06.768537 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8133574-1ee3-4378-8566-4997e93380c2 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:06.774604 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 371/748] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:51:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:06.774604 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26974d4-3261-4e77-83c7-eca594cbcd06 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:06.797743 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:06.810858 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:06.906673 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8133574-1ee3-4378-8566-4997e93380c2 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:06.917122 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26974d4-3261-4e77-83c7-eca594cbcd06 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:06.954132 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a094f7ad-c65b-4b67-816c-65b734f41698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:06.955423 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a094f7ad-c65b-4b67-816c-65b734f41698 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:06.955905 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a094f7ad-c65b-4b67-816c-65b734f41698 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:06.956548 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a094f7ad-c65b-4b67-816c-65b734f41698 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:06.962495 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107fe9dc-fb40-4412-969b-69aead7923a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:06.963105 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107fe9dc-fb40-4412-969b-69aead7923a9 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:06.963382 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107fe9dc-fb40-4412-969b-69aead7923a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:06.963651 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107fe9dc-fb40-4412-969b-69aead7923a9 None admin] PATH_INFO: `/v3/domains` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:06.966805 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107fe9dc-fb40-4412-969b-69aead7923a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:06.967113 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107fe9dc-fb40-4412-969b-69aead7923a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:06.969443 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107fe9dc-fb40-4412-969b-69aead7923a9 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:06.970894 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a094f7ad-c65b-4b67-816c-65b734f41698 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:06.973388 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a094f7ad-c65b-4b67-816c-65b734f41698 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:06.981733 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 372/749] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:51:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:06.982649 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107fe9dc-fb40-4412-969b-69aead7923a9 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:06.989028 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-a094f7ad-c65b-4b67-816c-65b734f41698 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:51:06.989028 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:06.989028 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:06.989028 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:06.989028 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:06.989028 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:06.989028 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:06.989028 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:06.989028 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:06.989028 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:06.989028 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:06.989028 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:06.989028 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:51:06.989028 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:51:06.989028 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:51:06.989028 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:51:06.990008 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:06.990008 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:06.990008 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:06.990008 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:06.990008 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:06.990008 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:06.990008 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:06.990008 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:06.990008 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:06.990008 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:06.990008 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:06.990008 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:06.990008 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:06.990008 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:06.990008 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:06.990008 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:06.990008 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:51:06.990963 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:51:06.990963 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:51:06.990963 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:06.990963 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 378/750] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:51:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:06.991650 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a094f7ad-c65b-4b67-816c-65b734f41698 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:07.078974 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:07.080582 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:07.107694 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107fe9dc-fb40-4412-969b-69aead7923a9 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:07.119918 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a094f7ad-c65b-4b67-816c-65b734f41698 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:07.145394 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f3f6d8-19c4-4a03-b336-444a8597338c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:07.146522 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f3f6d8-19c4-4a03-b336-444a8597338c None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:07.147011 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f3f6d8-19c4-4a03-b336-444a8597338c None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:07.147490 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f3f6d8-19c4-4a03-b336-444a8597338c None admin] PATH_INFO: `/v3/domains` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:07.148391 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f3f6d8-19c4-4a03-b336-444a8597338c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:07.148925 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f3f6d8-19c4-4a03-b336-444a8597338c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:07.150647 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f3f6d8-19c4-4a03-b336-444a8597338c None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:07.155886 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec92816-f150-4990-92ee-825c7dfa310c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:07.156662 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec92816-f150-4990-92ee-825c7dfa310c None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:07.156933 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec92816-f150-4990-92ee-825c7dfa310c None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:07.157203 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec92816-f150-4990-92ee-825c7dfa310c None admin] PATH_INFO: `/v3/users/swift` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:07.165860 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 373/751] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:51:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:07.166342 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec92816-f150-4990-92ee-825c7dfa310c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:07.167331 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f3f6d8-19c4-4a03-b336-444a8597338c None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:07.167790 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec92816-f150-4990-92ee-825c7dfa310c None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:07.180641 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-eec92816-f150-4990-92ee-825c7dfa310c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 06 21:51:07.180641 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:07.180641 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:07.180641 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:07.180641 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:07.180641 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:07.180641 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:07.180641 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:07.180641 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:07.180641 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:07.180641 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:07.180641 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:07.180641 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 21:51:07.180641 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 21:51:07.180641 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 21:51:07.180641 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 21:51:07.181651 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:07.181651 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:07.181651 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 21:51:07.181651 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 21:51:07.181651 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 21:51:07.181651 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 21:51:07.181651 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:07.181651 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:07.181651 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:07.181651 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:07.181651 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:07.181651 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:07.181651 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:07.181651 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:07.181651 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:07.181651 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:07.181651 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:07.181651 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:07.182678 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:07.182678 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:07.182678 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 21:51:07.182678 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 21:51:07.182678 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 21:51:07.182678 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 21:51:07.182678 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 21:51:07.182678 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 21:51:07.182678 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 06 21:51:07.182678 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:07.182678 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 379/752] 10.222.0.18 () {56 vars in 1096 bytes} [Sat Aug 6 21:51:06 2022] GET /identity/v3/users/swift => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:07.183272 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec92816-f150-4990-92ee-825c7dfa310c None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:07.279428 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f3f6d8-19c4-4a03-b336-444a8597338c None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:07.294941 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec92816-f150-4990-92ee-825c7dfa310c None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:07.318821 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4573314a-d9e1-4517-bc99-587130e93521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:07.319490 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4573314a-d9e1-4517-bc99-587130e93521 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:07.319705 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4573314a-d9e1-4517-bc99-587130e93521 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:07.319980 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4573314a-d9e1-4517-bc99-587130e93521 None admin] PATH_INFO: `/v3/domains` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:07.320669 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4573314a-d9e1-4517-bc99-587130e93521 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:07.320972 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4573314a-d9e1-4517-bc99-587130e93521 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:07.322338 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4573314a-d9e1-4517-bc99-587130e93521 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:07.324836 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:07.338014 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:07.339008 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 374/753] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:51:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:07.339991 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4573314a-d9e1-4517-bc99-587130e93521 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:07.340886 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874982cc-11ae-4158-bf01-0b59d5523e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:07.341591 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874982cc-11ae-4158-bf01-0b59d5523e2e None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:07.341948 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874982cc-11ae-4158-bf01-0b59d5523e2e None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:07.342297 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874982cc-11ae-4158-bf01-0b59d5523e2e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:07.362602 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874982cc-11ae-4158-bf01-0b59d5523e2e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:07.364430 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874982cc-11ae-4158-bf01-0b59d5523e2e None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:07.377234 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-874982cc-11ae-4158-bf01-0b59d5523e2e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 21:51:07.377234 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:07.377234 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:07.377234 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:07.377234 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:07.377234 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:07.377234 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:07.377234 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:07.377234 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:07.377234 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:07.377234 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:07.377234 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:07.377234 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 21:51:07.377234 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 21:51:07.377234 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 21:51:07.378188 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 21:51:07.378188 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:07.378188 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:07.378188 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:07.378188 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:07.378188 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:07.378188 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:07.378188 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:07.378188 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:07.378188 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:07.378188 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:07.378188 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:07.378188 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:07.378188 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:07.378188 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:07.378188 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:07.378188 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:07.379095 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 21:51:07.379095 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 21:51:07.379095 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 21:51:07.379095 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 21:51:07.379095 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 21:51:07.379095 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 21:51:07.379095 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 21:51:07.379095 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:07.381513 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 380/754] 10.222.0.18 () {56 vars in 1111 bytes} [Sat Aug 6 21:51:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:07.382519 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874982cc-11ae-4158-bf01-0b59d5523e2e None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:07.455723 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4573314a-d9e1-4517-bc99-587130e93521 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:07.486561 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874982cc-11ae-4158-bf01-0b59d5523e2e None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:07.493464 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94d0e9b-e7f2-4209-8b35-83a08f6b6a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:07.494080 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d0e9b-e7f2-4209-8b35-83a08f6b6a94 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:07.494360 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d0e9b-e7f2-4209-8b35-83a08f6b6a94 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:07.495005 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d0e9b-e7f2-4209-8b35-83a08f6b6a94 None admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:07.495359 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94d0e9b-e7f2-4209-8b35-83a08f6b6a94 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:07.495664 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94d0e9b-e7f2-4209-8b35-83a08f6b6a94 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:07.497277 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94d0e9b-e7f2-4209-8b35-83a08f6b6a94 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:07.517168 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 375/755] 10.222.0.18 () {56 vars in 1135 bytes} [Sat Aug 6 21:51:07 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:07.518104 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94d0e9b-e7f2-4209-8b35-83a08f6b6a94 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:07.525560 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbf087c-3954-473d-8215-e989e72052d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:07.526581 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf087c-3954-473d-8215-e989e72052d6 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:07.527067 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf087c-3954-473d-8215-e989e72052d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:07.527546 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf087c-3954-473d-8215-e989e72052d6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:07.535694 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbf087c-3954-473d-8215-e989e72052d6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:07.537503 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbf087c-3954-473d-8215-e989e72052d6 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:07.549827 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-4fbf087c-3954-473d-8215-e989e72052d6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 21:51:07.549827 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:07.549827 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:07.549827 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:07.549827 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:07.549827 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:07.549827 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:07.549827 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:07.549827 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:07.549827 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:07.549827 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:07.549827 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:07.549827 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 21:51:07.549827 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 21:51:07.549827 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 21:51:07.550773 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 21:51:07.550773 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:07.550773 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:07.550773 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:07.550773 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:07.550773 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:07.550773 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:07.550773 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:07.550773 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:07.550773 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:07.550773 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:07.550773 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:07.550773 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:07.550773 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:07.550773 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:07.550773 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:07.550773 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:07.551694 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 21:51:07.551694 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 21:51:07.551694 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 21:51:07.551694 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 21:51:07.551694 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 21:51:07.551694 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 21:51:07.551694 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 21:51:07.551694 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:07.554090 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 381/756] 10.222.0.18 () {56 vars in 1111 bytes} [Sat Aug 6 21:51:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:07.555102 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbf087c-3954-473d-8215-e989e72052d6 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:07.576743 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:07.632272 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94d0e9b-e7f2-4209-8b35-83a08f6b6a94 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:07.632813 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:07.675044 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1bb588-3d30-4cb7-a5b4-401984145f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:07.676082 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1bb588-3d30-4cb7-a5b4-401984145f31 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:07.676367 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1bb588-3d30-4cb7-a5b4-401984145f31 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:07.676910 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1bb588-3d30-4cb7-a5b4-401984145f31 None admin] PATH_INFO: `/v3/projects` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:07.677622 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1bb588-3d30-4cb7-a5b4-401984145f31 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:07.677932 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1bb588-3d30-4cb7-a5b4-401984145f31 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:07.678611 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbf087c-3954-473d-8215-e989e72052d6 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:07.679364 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1bb588-3d30-4cb7-a5b4-401984145f31 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:07.700501 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 376/757] 10.222.0.18 () {56 vars in 1148 bytes} [Sat Aug 6 21:51:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:07.701376 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1bb588-3d30-4cb7-a5b4-401984145f31 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:07.721276 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cc5294-6510-403d-96f8-4b3bf5b62f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:07.721880 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cc5294-6510-403d-96f8-4b3bf5b62f07 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:07.722160 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cc5294-6510-403d-96f8-4b3bf5b62f07 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:07.722433 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cc5294-6510-403d-96f8-4b3bf5b62f07 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:07.802939 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cc5294-6510-403d-96f8-4b3bf5b62f07 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:07.805342 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cc5294-6510-403d-96f8-4b3bf5b62f07 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:07.828186 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1bb588-3d30-4cb7-a5b4-401984145f31 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:07.845431 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:07.862259 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:07.873262 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baf3074-b327-43d7-b48d-e7545eb801ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:07.874101 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baf3074-b327-43d7-b48d-e7545eb801ce None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:07.875123 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baf3074-b327-43d7-b48d-e7545eb801ce None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:07.875576 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baf3074-b327-43d7-b48d-e7545eb801ce None admin] PATH_INFO: `/v3/projects` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:07.876472 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baf3074-b327-43d7-b48d-e7545eb801ce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:07.876923 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baf3074-b327-43d7-b48d-e7545eb801ce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:07.878622 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baf3074-b327-43d7-b48d-e7545eb801ce None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:07.892972 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 377/758] 10.222.0.18 () {56 vars in 1148 bytes} [Sat Aug 6 21:51:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:07.893587 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baf3074-b327-43d7-b48d-e7545eb801ce None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:07.927995 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 382/759] 10.222.0.18 () {58 vars in 1282 bytes} [Sat Aug 6 21:51:07 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 21:51:07.927995 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cc5294-6510-403d-96f8-4b3bf5b62f07 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:08.011553 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baf3074-b327-43d7-b48d-e7545eb801ce None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:08.037847 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cc5294-6510-403d-96f8-4b3bf5b62f07 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:08.049852 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e061c209-3277-48b2-859d-be037dca8d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:08.050519 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e061c209-3277-48b2-859d-be037dca8d8e None admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:08.052519 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e061c209-3277-48b2-859d-be037dca8d8e None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:08.053058 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e061c209-3277-48b2-859d-be037dca8d8e None admin] PATH_INFO: `/v3/projects/4bb6307eb0644d85a1d1daaa61352697/users/2a1319d33e9144859c6010843be312a3/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:08.057008 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e061c209-3277-48b2-859d-be037dca8d8e None admin] RBAC: Authorizing `identity:create_grant(project_id=4bb6307eb0644d85a1d1daaa61352697, user_id=2a1319d33e9144859c6010843be312a3, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:08.058465 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e061c209-3277-48b2-859d-be037dca8d8e None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:08.083788 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b276c95-ffc3-45b0-bd2f-c8e6a45802c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:08.084956 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b276c95-ffc3-45b0-bd2f-c8e6a45802c4 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:08.085482 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b276c95-ffc3-45b0-bd2f-c8e6a45802c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:08.085981 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b276c95-ffc3-45b0-bd2f-c8e6a45802c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:08.086943 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b276c95-ffc3-45b0-bd2f-c8e6a45802c4 None admin] RBAC: Adding query filter params (role.id=d90649d4ef604eeb8edcf0aa22932ca1, scope.project.id=4bb6307eb0644d85a1d1daaa61352697, user.id=d1b26ba592724dd389584bb800d877a2) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:08.087513 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b276c95-ffc3-45b0-bd2f-c8e6a45802c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:08.089224 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b276c95-ffc3-45b0-bd2f-c8e6a45802c4 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:08.096107 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 383/760] 10.222.0.18 () {56 vars in 1374 bytes} [Sat Aug 6 21:51:07 2022] GET /identity/v3/role_assignments?user.id=d1b26ba592724dd389584bb800d877a2&scope.project.id=4bb6307eb0644d85a1d1daaa61352697&role.id=d90649d4ef604eeb8edcf0aa22932ca1 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:08.097282 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b276c95-ffc3-45b0-bd2f-c8e6a45802c4 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:08.116565 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:08.180839 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 378/761] 10.222.0.18 () {58 vars in 1439 bytes} [Sat Aug 6 21:51:07 2022] PUT /identity/v3/projects/4bb6307eb0644d85a1d1daaa61352697/users/2a1319d33e9144859c6010843be312a3/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 21:51:08.180839 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f98d3f-54fe-48c5-8bcb-2a25868ac93c None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:08.180839 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f98d3f-54fe-48c5-8bcb-2a25868ac93c None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:08.180839 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f98d3f-54fe-48c5-8bcb-2a25868ac93c None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:08.180839 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 379/762] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:51:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:51:08.194199 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5386358e-0817-4368-ad61-dceee34f0a10 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:08.194199 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5386358e-0817-4368-ad61-dceee34f0a10 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:08.194199 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5386358e-0817-4368-ad61-dceee34f0a10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:08.229535 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-5386358e-0817-4368-ad61-dceee34f0a10 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:51:08.230785 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-5386358e-0817-4368-ad61-dceee34f0a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:51:08.332523 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b276c95-ffc3-45b0-bd2f-c8e6a45802c4 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:08.347819 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:08.428126 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5ffd4f-8da7-49e8-9c5c-d3d7bf6788bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:08.428793 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5ffd4f-8da7-49e8-9c5c-d3d7bf6788bb None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:08.429077 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5ffd4f-8da7-49e8-9c5c-d3d7bf6788bb None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:08.429343 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5ffd4f-8da7-49e8-9c5c-d3d7bf6788bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:08.448044 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 380/763] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:51:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 21:51:08.448216 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5ffd4f-8da7-49e8-9c5c-d3d7bf6788bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:08.452504 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5ffd4f-8da7-49e8-9c5c-d3d7bf6788bb None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:08.467491 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-8e5ffd4f-8da7-49e8-9c5c-d3d7bf6788bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:51:08.467491 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:08.467491 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:08.467491 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:08.467491 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:08.467491 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:08.467491 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:08.467491 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:08.467491 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:08.467491 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:08.467491 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:08.467491 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:08.467491 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:51:08.467491 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:51:08.467491 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:51:08.467491 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:51:08.468538 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:08.468538 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:08.468538 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:08.468538 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:08.468538 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:08.468538 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:08.468538 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:08.468538 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:08.468538 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:08.468538 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:08.468538 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:08.468538 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:08.468538 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:08.468538 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:08.468538 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:08.468538 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:08.468538 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:51:08.469426 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:51:08.469426 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:51:08.469426 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:08.471487 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 384/764] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:51:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 374 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:08.475374 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5386358e-0817-4368-ad61-dceee34f0a10 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:08.486091 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5ffd4f-8da7-49e8-9c5c-d3d7bf6788bb None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:08.601329 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:08.694603 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5ffd4f-8da7-49e8-9c5c-d3d7bf6788bb None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:08.703252 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5386358e-0817-4368-ad61-dceee34f0a10 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:08.743147 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3246a249-7a56-403f-bd1d-9cea7f794760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:08.744077 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3246a249-7a56-403f-bd1d-9cea7f794760 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:08.744577 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3246a249-7a56-403f-bd1d-9cea7f794760 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:08.745026 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3246a249-7a56-403f-bd1d-9cea7f794760 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:08.745947 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3246a249-7a56-403f-bd1d-9cea7f794760 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:08.748011 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3246a249-7a56-403f-bd1d-9cea7f794760 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:08.757885 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-3246a249-7a56-403f-bd1d-9cea7f794760 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 06 21:51:08.757885 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:08.757885 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 21:51:08.757885 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 21:51:08.757885 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 21:51:08.757885 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 21:51:08.757885 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 21:51:08.757885 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 21:51:08.757885 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 06 21:51:08.757885 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:08.757885 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 21:51:08.757885 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:08.757885 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:08.757885 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:08.757885 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:08.757885 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:08.757885 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:08.758870 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:08.758870 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:08.758870 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:08.758870 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:08.758870 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:08.758870 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:08.758870 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 21:51:08.758870 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 21:51:08.758870 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 21:51:08.758870 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 21:51:08.758870 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:08.758870 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:08.758870 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:08.758870 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:08.758870 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:08.758870 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:08.759695 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:08.759695 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:08.759695 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:08.759695 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:08.759695 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:08.759695 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:08.759695 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:08.759695 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:08.759695 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:08.759695 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:08.759695 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 21:51:08.759695 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 21:51:08.759695 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 06 21:51:08.759695 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:08.762996 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 381/765] 10.222.0.18 () {56 vars in 1129 bytes} [Sat Aug 6 21:51:08 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:08.769458 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3246a249-7a56-403f-bd1d-9cea7f794760 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:08.769682 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1ebfa1-2576-453e-b230-5ce0db758d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:08.776457 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ebfa1-2576-453e-b230-5ce0db758d21 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:08.776457 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ebfa1-2576-453e-b230-5ce0db758d21 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:08.776457 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ebfa1-2576-453e-b230-5ce0db758d21 None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:08.776457 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1ebfa1-2576-453e-b230-5ce0db758d21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:08.776457 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1ebfa1-2576-453e-b230-5ce0db758d21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:08.776457 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1ebfa1-2576-453e-b230-5ce0db758d21 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:08.793862 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 385/766] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:51:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:51:08.793862 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1ebfa1-2576-453e-b230-5ce0db758d21 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:08.826515 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:08.933264 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1ebfa1-2576-453e-b230-5ce0db758d21 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:08.948585 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3246a249-7a56-403f-bd1d-9cea7f794760 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:08.964395 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:08.980372 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6bfad4-57e2-4951-bf98-e04e71cd4f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:08.981497 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6bfad4-57e2-4951-bf98-e04e71cd4f7a None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:08.982005 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6bfad4-57e2-4951-bf98-e04e71cd4f7a None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:08.982799 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6bfad4-57e2-4951-bf98-e04e71cd4f7a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:08.984863 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2723ddd-8c26-4751-bb67-4a298efddb61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:08.985579 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2723ddd-8c26-4751-bb67-4a298efddb61 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:08.986021 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2723ddd-8c26-4751-bb67-4a298efddb61 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:08.986387 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2723ddd-8c26-4751-bb67-4a298efddb61 None admin] PATH_INFO: `/v3/services` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:08.987190 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2723ddd-8c26-4751-bb67-4a298efddb61 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:08.987780 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2723ddd-8c26-4751-bb67-4a298efddb61 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:08.990041 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2723ddd-8c26-4751-bb67-4a298efddb61 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:08.993608 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6bfad4-57e2-4951-bf98-e04e71cd4f7a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:08.995194 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6bfad4-57e2-4951-bf98-e04e71cd4f7a None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:09.000624 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 382/767] 10.222.0.18 () {56 vars in 1124 bytes} [Sat Aug 6 21:51:08 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:09.001930 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041d51b8-e317-4d2d-b231-e7b39b072aa2 None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:09.002213 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041d51b8-e317-4d2d-b231-e7b39b072aa2 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:09.002486 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041d51b8-e317-4d2d-b231-e7b39b072aa2 None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:09.004121 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 383/768] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:51:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:51:09.004859 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041d51b8-e317-4d2d-b231-e7b39b072aa2 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:09.009561 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-cb6bfad4-57e2-4951-bf98-e04e71cd4f7a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 21:51:09.009561 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:09.009561 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:09.009561 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:09.009561 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:09.009561 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:09.009561 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:09.009561 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:09.009561 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:09.009561 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:09.009561 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:09.009561 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:09.009561 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 21:51:09.009561 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 21:51:09.009561 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 21:51:09.010461 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 21:51:09.010461 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:09.010461 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:09.010461 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:09.010461 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:09.010461 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:09.010461 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:09.010461 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:09.010461 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:09.010461 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:09.010461 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:09.010461 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:09.010461 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:09.010461 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:09.010461 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:09.010461 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:09.010461 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:09.011356 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 21:51:09.011356 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 21:51:09.011356 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 21:51:09.011356 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 21:51:09.011356 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 21:51:09.011356 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 21:51:09.011356 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 21:51:09.011356 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:09.018826 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 386/769] 10.222.0.18 () {56 vars in 1111 bytes} [Sat Aug 6 21:51:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:09.020302 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b476ba05-763b-40a8-bdfb-0a610dce76cf None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:09.021150 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b476ba05-763b-40a8-bdfb-0a610dce76cf None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:09.021594 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b476ba05-763b-40a8-bdfb-0a610dce76cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:09.051329 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-b476ba05-763b-40a8-bdfb-0a610dce76cf None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:51:09.052904 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-b476ba05-763b-40a8-bdfb-0a610dce76cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:51:09.123783 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:09.130283 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 387/770] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:51:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 21:51:09.138090 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b476ba05-763b-40a8-bdfb-0a610dce76cf None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:09.138208 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041d51b8-e317-4d2d-b231-e7b39b072aa2 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:09.179479 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4195f46-8270-4901-a11b-80c94c352e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:09.181032 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4195f46-8270-4901-a11b-80c94c352e81 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:09.181484 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4195f46-8270-4901-a11b-80c94c352e81 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:09.181917 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4195f46-8270-4901-a11b-80c94c352e81 None admin] PATH_INFO: `/v3/services` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:09.182914 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4195f46-8270-4901-a11b-80c94c352e81 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:09.183672 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4195f46-8270-4901-a11b-80c94c352e81 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:09.185632 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4195f46-8270-4901-a11b-80c94c352e81 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:09.191944 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 384/771] 10.222.0.18 () {56 vars in 1124 bytes} [Sat Aug 6 21:51:09 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:09.193106 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4195f46-8270-4901-a11b-80c94c352e81 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:09.266673 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b476ba05-763b-40a8-bdfb-0a610dce76cf None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:09.282768 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:09.310140 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e0d9d8-179f-4c0d-88bd-ef1febd13a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:09.310833 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e0d9d8-179f-4c0d-88bd-ef1febd13a7a None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:09.311198 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e0d9d8-179f-4c0d-88bd-ef1febd13a7a None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:09.312001 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e0d9d8-179f-4c0d-88bd-ef1febd13a7a None admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:09.313063 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e0d9d8-179f-4c0d-88bd-ef1febd13a7a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:09.313437 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e0d9d8-179f-4c0d-88bd-ef1febd13a7a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:09.315796 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e0d9d8-179f-4c0d-88bd-ef1febd13a7a None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:09.319429 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4195f46-8270-4901-a11b-80c94c352e81 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:09.332487 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 388/772] 10.222.0.18 () {56 vars in 1148 bytes} [Sat Aug 6 21:51:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:09.333553 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e0d9d8-179f-4c0d-88bd-ef1febd13a7a None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:09.378764 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e75af60-5028-40d8-8470-c8c597e65cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:09.379547 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e75af60-5028-40d8-8470-c8c597e65cbc None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:09.379927 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e75af60-5028-40d8-8470-c8c597e65cbc None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:09.380303 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e75af60-5028-40d8-8470-c8c597e65cbc None admin] PATH_INFO: `/v3/services/network` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:09.381265 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e75af60-5028-40d8-8470-c8c597e65cbc None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:09.383800 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e75af60-5028-40d8-8470-c8c597e65cbc None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:09.396338 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-6e75af60-5028-40d8-8470-c8c597e65cbc None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 06 21:51:09.396338 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:09.396338 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 21:51:09.396338 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 21:51:09.396338 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 21:51:09.396338 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 21:51:09.396338 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 21:51:09.396338 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 21:51:09.396338 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 06 21:51:09.396338 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:09.396338 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 21:51:09.396338 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:09.396338 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:09.396338 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:09.396338 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:09.396338 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:09.396338 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:09.397497 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:09.397497 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:09.397497 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:09.397497 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:09.397497 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:09.397497 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:09.397497 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 21:51:09.397497 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 21:51:09.397497 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 21:51:09.397497 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 21:51:09.397497 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:09.397497 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:09.397497 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:09.397497 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:09.397497 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:09.397497 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:09.398345 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:09.398345 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:09.398345 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:09.398345 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:09.398345 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:09.398345 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:09.398345 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:09.398345 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:09.398345 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:09.398345 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:09.398345 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 21:51:09.398345 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 21:51:09.398345 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 06 21:51:09.398345 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:09.398345 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 385/773] 10.222.0.18 () {56 vars in 1111 bytes} [Sat Aug 6 21:51:09 2022] GET /identity/v3/services/network => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:09.404302 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e75af60-5028-40d8-8470-c8c597e65cbc None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:09.427085 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:09.484667 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e0d9d8-179f-4c0d-88bd-ef1febd13a7a None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:09.518534 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e75af60-5028-40d8-8470-c8c597e65cbc None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:09.527644 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef74a52-49cf-44fa-9bf6-a8a8cc0bb1c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:09.528532 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef74a52-49cf-44fa-9bf6-a8a8cc0bb1c7 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:09.529344 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef74a52-49cf-44fa-9bf6-a8a8cc0bb1c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:09.529823 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef74a52-49cf-44fa-9bf6-a8a8cc0bb1c7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:09.531654 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef74a52-49cf-44fa-9bf6-a8a8cc0bb1c7 None admin] RBAC: Adding query filter params (interface=public, service_id=5746cbcc26e440588b91e53d803c47d1, region_id=RegionOne) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:09.532307 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef74a52-49cf-44fa-9bf6-a8a8cc0bb1c7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:09.534184 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef74a52-49cf-44fa-9bf6-a8a8cc0bb1c7 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:09.550167 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 389/774] 10.222.0.18 () {56 vars in 1251 bytes} [Sat Aug 6 21:51:09 2022] GET /identity/v3/endpoints?service_id=5746cbcc26e440588b91e53d803c47d1&interface=public®ion_id=RegionOne => generated 194 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:09.550167 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef74a52-49cf-44fa-9bf6-a8a8cc0bb1c7 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:09.562693 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ade545-5488-4218-b8a7-4aaca620d7d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:09.563409 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ade545-5488-4218-b8a7-4aaca620d7d0 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:09.563693 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ade545-5488-4218-b8a7-4aaca620d7d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:09.563966 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ade545-5488-4218-b8a7-4aaca620d7d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:09.564737 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ade545-5488-4218-b8a7-4aaca620d7d0 None admin] RBAC: Adding query filter params (role.id=4b8dcf0491fd4072acc69fd894bf77d0, scope.project.id=4bb6307eb0644d85a1d1daaa61352697, user.id=e7fc9da7505744608dfd3bbd46a0a9d8) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:09.565054 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ade545-5488-4218-b8a7-4aaca620d7d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:09.566549 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ade545-5488-4218-b8a7-4aaca620d7d0 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:09.580578 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 386/775] 10.222.0.18 () {56 vars in 1374 bytes} [Sat Aug 6 21:51:09 2022] GET /identity/v3/role_assignments?user.id=e7fc9da7505744608dfd3bbd46a0a9d8&scope.project.id=4bb6307eb0644d85a1d1daaa61352697&role.id=4b8dcf0491fd4072acc69fd894bf77d0 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:09.581624 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ade545-5488-4218-b8a7-4aaca620d7d0 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:09.597479 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:09.679597 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef74a52-49cf-44fa-9bf6-a8a8cc0bb1c7 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:09.724903 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8958ea-5691-406b-bab6-738ab7d355c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:09.726033 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8958ea-5691-406b-bab6-738ab7d355c7 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:09.726567 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8958ea-5691-406b-bab6-738ab7d355c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:09.727040 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8958ea-5691-406b-bab6-738ab7d355c7 None admin] PATH_INFO: `/v3/services` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:09.728222 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8958ea-5691-406b-bab6-738ab7d355c7 None admin] RBAC: Adding query filter params (name=network) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:09.728740 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8958ea-5691-406b-bab6-738ab7d355c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:09.730568 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8958ea-5691-406b-bab6-738ab7d355c7 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:09.737585 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 390/776] 10.222.0.18 () {56 vars in 1112 bytes} [Sat Aug 6 21:51:09 2022] GET /identity/v3/services?name=network => generated 124 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:09.744280 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8958ea-5691-406b-bab6-738ab7d355c7 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:09.798036 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:09.831386 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ade545-5488-4218-b8a7-4aaca620d7d0 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:09.870222 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:09.874074 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8958ea-5691-406b-bab6-738ab7d355c7 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:09.895364 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c02d46-1273-47a3-be8c-45f9c35eab52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:09.896309 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c02d46-1273-47a3-be8c-45f9c35eab52 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:09.896748 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c02d46-1273-47a3-be8c-45f9c35eab52 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:09.897146 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c02d46-1273-47a3-be8c-45f9c35eab52 None admin] PATH_INFO: `/v3/services` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:09.897893 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c02d46-1273-47a3-be8c-45f9c35eab52 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:09.902002 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c02d46-1273-47a3-be8c-45f9c35eab52 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:09.920650 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 387/777] 10.222.0.18 () {56 vars in 1087 bytes} [Sat Aug 6 21:51:09 2022] GET /identity/v3/services => generated 772 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:09.936640 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09caae79-8e9b-41b2-a9b3-4992c0c4cd2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:09.938052 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09caae79-8e9b-41b2-a9b3-4992c0c4cd2a None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:09.938471 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09caae79-8e9b-41b2-a9b3-4992c0c4cd2a None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:09.938902 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09caae79-8e9b-41b2-a9b3-4992c0c4cd2a None admin] PATH_INFO: `/v3/services` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:09.939675 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09caae79-8e9b-41b2-a9b3-4992c0c4cd2a None admin] RBAC: Adding query filter params (type=network) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:09.940117 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09caae79-8e9b-41b2-a9b3-4992c0c4cd2a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:09.941654 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09caae79-8e9b-41b2-a9b3-4992c0c4cd2a None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:09.952140 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 391/778] 10.222.0.18 () {56 vars in 1112 bytes} [Sat Aug 6 21:51:09 2022] GET /identity/v3/services?type=network => generated 350 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:09.958516 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c02d46-1273-47a3-be8c-45f9c35eab52 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:10.102663 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c02d46-1273-47a3-be8c-45f9c35eab52 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:10.143048 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dfd8fb-70c1-4c88-9ebb-1e1c508a817e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:10.143672 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dfd8fb-70c1-4c88-9ebb-1e1c508a817e None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:10.144000 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dfd8fb-70c1-4c88-9ebb-1e1c508a817e None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:10.144274 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dfd8fb-70c1-4c88-9ebb-1e1c508a817e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:10.145026 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dfd8fb-70c1-4c88-9ebb-1e1c508a817e None admin] RBAC: Adding query filter params (interface=public, service_id=cdf55d5830104e3cb18d9d2a801727e4, region_id=RegionOne) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:10.145332 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dfd8fb-70c1-4c88-9ebb-1e1c508a817e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:10.146812 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dfd8fb-70c1-4c88-9ebb-1e1c508a817e None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:10.156055 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 388/779] 10.222.0.18 () {56 vars in 1251 bytes} [Sat Aug 6 21:51:09 2022] GET /identity/v3/endpoints?service_id=cdf55d5830104e3cb18d9d2a801727e4&interface=public®ion_id=RegionOne => generated 194 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:10.161940 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09caae79-8e9b-41b2-a9b3-4992c0c4cd2a None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:10.289540 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09caae79-8e9b-41b2-a9b3-4992c0c4cd2a None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:10.329852 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05546d3-be97-4f02-9c88-ac386093b497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:10.330552 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05546d3-be97-4f02-9c88-ac386093b497 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:10.330898 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05546d3-be97-4f02-9c88-ac386093b497 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:10.331235 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05546d3-be97-4f02-9c88-ac386093b497 None admin] PATH_INFO: `/v3/services` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:10.331901 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05546d3-be97-4f02-9c88-ac386093b497 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:10.333394 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05546d3-be97-4f02-9c88-ac386093b497 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:10.340454 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 392/780] 10.222.0.18 () {56 vars in 1087 bytes} [Sat Aug 6 21:51:10 2022] GET /identity/v3/services => generated 772 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:10.431189 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553367c8-4a99-4eb3-9f07-561101c11a1e None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:10.431695 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553367c8-4a99-4eb3-9f07-561101c11a1e None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:10.431695 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553367c8-4a99-4eb3-9f07-561101c11a1e None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:10.433373 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 389/781] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:51:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:51:10.439548 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4fe066-0f5c-4a58-a060-2c18ff34e4e9 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:10.439835 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4fe066-0f5c-4a58-a060-2c18ff34e4e9 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:10.440120 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4fe066-0f5c-4a58-a060-2c18ff34e4e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:10.466043 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-bc4fe066-0f5c-4a58-a060-2c18ff34e4e9 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:51:10.467278 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-bc4fe066-0f5c-4a58-a060-2c18ff34e4e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:51:10.506383 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:10.559517 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823f7155-5499-40a6-a93b-2f1684da8219 None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:10.559806 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823f7155-5499-40a6-a93b-2f1684da8219 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:10.560087 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823f7155-5499-40a6-a93b-2f1684da8219 None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:10.561695 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 390/782] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:51:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:51:10.567173 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea488ab8-7ec9-4ac0-b7a1-e48963297b02 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:10.567453 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea488ab8-7ec9-4ac0-b7a1-e48963297b02 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:10.567726 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea488ab8-7ec9-4ac0-b7a1-e48963297b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:10.572246 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 393/783] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:51:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 21:51:10.591425 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-ea488ab8-7ec9-4ac0-b7a1-e48963297b02 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:51:10.592658 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-ea488ab8-7ec9-4ac0-b7a1-e48963297b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:51:10.597107 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:10.607744 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4fe066-0f5c-4a58-a060-2c18ff34e4e9 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:10.672394 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 391/784] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:51:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 21:51:10.704242 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea488ab8-7ec9-4ac0-b7a1-e48963297b02 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:10.748152 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4fe066-0f5c-4a58-a060-2c18ff34e4e9 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:10.791556 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3fca99-1671-4084-8d7d-c65690d0c9e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:10.792567 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3fca99-1671-4084-8d7d-c65690d0c9e4 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:10.793296 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3fca99-1671-4084-8d7d-c65690d0c9e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:10.794986 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3fca99-1671-4084-8d7d-c65690d0c9e4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:10.817623 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3fca99-1671-4084-8d7d-c65690d0c9e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:51:10.823272 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-1c3fca99-1671-4084-8d7d-c65690d0c9e4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 21:51:10.823272 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:10.823272 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:10.823272 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:10.823272 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:10.823272 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:10.823272 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:10.823272 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:10.823272 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:10.823272 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:10.823272 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:10.823272 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:10.823272 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:51:10.823272 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:51:10.823272 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:51:10.823272 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:51:10.824194 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:51:10.824194 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:51:10.824194 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:51:10.824194 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:51:10.824194 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:10.824194 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:10.824194 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:10.824194 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:10.824194 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:10.824194 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:10.824194 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:10.824194 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:10.824194 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:10.824194 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:10.824194 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:10.824194 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:10.825050 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:10.825050 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:10.825050 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:10.825050 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:10.825050 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:51:10.825050 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:51:10.825050 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:51:10.825050 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:51:10.825050 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:51:10.825050 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:51:10.825050 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 21:51:10.825050 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:10.831082 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 394/785] 10.222.0.18 () {56 vars in 1102 bytes} [Sat Aug 6 21:51:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 21:51:10.837187 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3fca99-1671-4084-8d7d-c65690d0c9e4 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:10.861427 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:10.902711 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea488ab8-7ec9-4ac0-b7a1-e48963297b02 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:10.972799 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797c5496-82ad-42c1-a970-e975ffd16a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:10.973469 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797c5496-82ad-42c1-a970-e975ffd16a8c None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:10.973748 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797c5496-82ad-42c1-a970-e975ffd16a8c None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:10.974018 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797c5496-82ad-42c1-a970-e975ffd16a8c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:10.982493 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:10.986710 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797c5496-82ad-42c1-a970-e975ffd16a8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:51:10.991733 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-797c5496-82ad-42c1-a970-e975ffd16a8c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 21:51:10.991733 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:10.991733 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:10.991733 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:10.991733 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:10.991733 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:10.991733 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:10.991733 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:10.991733 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:10.991733 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:10.991733 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:10.991733 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:10.991733 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:51:10.991733 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:51:10.991733 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:51:10.991733 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:51:10.997598 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:51:10.997598 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:51:10.997598 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:51:10.997598 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:51:10.997598 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:10.997598 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:10.997598 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:10.997598 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:10.997598 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:10.997598 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:10.997598 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:10.997598 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:10.997598 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:10.997598 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:10.997598 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:10.997598 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:10.998429 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:10.998429 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:10.998429 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:10.998429 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:10.998429 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:51:10.998429 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:51:10.998429 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:51:10.998429 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:51:10.998429 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:51:10.998429 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:51:10.998429 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 21:51:10.998429 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:10.998429 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 392/786] 10.222.0.18 () {56 vars in 1096 bytes} [Sat Aug 6 21:51:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 21:51:11.002311 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797c5496-82ad-42c1-a970-e975ffd16a8c None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:11.011941 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3fca99-1671-4084-8d7d-c65690d0c9e4 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:11.083793 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dc7985-e292-4a44-9c76-47110431d209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:11.084897 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dc7985-e292-4a44-9c76-47110431d209 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:11.085374 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dc7985-e292-4a44-9c76-47110431d209 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:11.085857 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dc7985-e292-4a44-9c76-47110431d209 None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:11.086709 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dc7985-e292-4a44-9c76-47110431d209 None admin] RBAC: Adding query filter params (name=service) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:11.087202 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dc7985-e292-4a44-9c76-47110431d209 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:11.088806 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dc7985-e292-4a44-9c76-47110431d209 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:11.099342 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 395/787] 10.222.0.18 () {56 vars in 1103 bytes} [Sat Aug 6 21:51:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:11.109972 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dc7985-e292-4a44-9c76-47110431d209 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:11.161586 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797c5496-82ad-42c1-a970-e975ffd16a8c None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:11.227795 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1c00df-73dc-4ef8-91d0-56e3d5acd13f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:11.228325 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1c00df-73dc-4ef8-91d0-56e3d5acd13f None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:11.228622 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1c00df-73dc-4ef8-91d0-56e3d5acd13f None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:11.228892 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1c00df-73dc-4ef8-91d0-56e3d5acd13f None admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:11.229563 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1c00df-73dc-4ef8-91d0-56e3d5acd13f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:11.229858 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1c00df-73dc-4ef8-91d0-56e3d5acd13f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:11.231234 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1c00df-73dc-4ef8-91d0-56e3d5acd13f None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:11.249699 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 393/788] 10.222.0.18 () {56 vars in 1099 bytes} [Sat Aug 6 21:51:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:11.256531 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1c00df-73dc-4ef8-91d0-56e3d5acd13f None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:11.257356 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:11.290566 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dc7985-e292-4a44-9c76-47110431d209 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:11.323396 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:11.363823 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1c00df-73dc-4ef8-91d0-56e3d5acd13f None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:11.373120 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6b48a1-8506-4694-8275-9032b3009faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:11.373778 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6b48a1-8506-4694-8275-9032b3009faf None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:11.374060 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6b48a1-8506-4694-8275-9032b3009faf None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:11.375187 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6b48a1-8506-4694-8275-9032b3009faf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:11.390036 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6b48a1-8506-4694-8275-9032b3009faf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:11.391513 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6b48a1-8506-4694-8275-9032b3009faf None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:11.400562 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2888a27-1b29-4b3a-8f4c-2c70b0c7f99a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:11.401192 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2888a27-1b29-4b3a-8f4c-2c70b0c7f99a None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:11.401460 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2888a27-1b29-4b3a-8f4c-2c70b0c7f99a None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:11.401728 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2888a27-1b29-4b3a-8f4c-2c70b0c7f99a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:11.411516 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-7f6b48a1-8506-4694-8275-9032b3009faf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:51:11.411516 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:11.411516 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:11.411516 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:11.411516 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:11.411516 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:11.411516 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:11.411516 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:11.411516 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:11.411516 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:11.411516 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:11.411516 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:11.411516 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:51:11.411516 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:51:11.411516 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:51:11.411516 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:51:11.412623 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:11.412623 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:11.412623 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:11.412623 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:11.412623 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:11.412623 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:11.412623 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:11.412623 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:11.412623 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:11.412623 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:11.412623 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:11.412623 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:11.412623 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:11.412623 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:11.412623 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:11.412623 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:11.412623 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:51:11.413667 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:51:11.413667 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:51:11.413667 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:11.420446 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2888a27-1b29-4b3a-8f4c-2c70b0c7f99a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:11.420627 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 396/789] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:51:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 21:51:11.420721 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2888a27-1b29-4b3a-8f4c-2c70b0c7f99a None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:11.420980 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6b48a1-8506-4694-8275-9032b3009faf None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:11.449940 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-a2888a27-1b29-4b3a-8f4c-2c70b0c7f99a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:51:11.449940 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:11.449940 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:11.449940 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:11.449940 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:11.449940 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:11.449940 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:11.449940 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:11.449940 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:11.449940 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:11.449940 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:11.449940 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:11.449940 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:51:11.449940 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:51:11.449940 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:51:11.449940 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:51:11.451319 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:11.451319 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:11.451319 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:11.451319 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:11.451319 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:11.451319 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:11.451319 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:11.451319 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:11.451319 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:11.451319 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:11.451319 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:11.451319 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:11.451319 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:11.451319 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:11.451319 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:11.451319 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:11.451319 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:51:11.452198 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:51:11.452198 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:51:11.452198 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:11.452198 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 394/790] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:51:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:11.457474 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2888a27-1b29-4b3a-8f4c-2c70b0c7f99a None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:11.588574 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6b48a1-8506-4694-8275-9032b3009faf None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:11.636237 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2888a27-1b29-4b3a-8f4c-2c70b0c7f99a None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:11.650082 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:11.659450 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a123bb34-21c2-43e4-9182-e8288a2003cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:11.660063 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a123bb34-21c2-43e4-9182-e8288a2003cb None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:11.660341 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a123bb34-21c2-43e4-9182-e8288a2003cb None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:11.663607 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a123bb34-21c2-43e4-9182-e8288a2003cb None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:11.663607 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a123bb34-21c2-43e4-9182-e8288a2003cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:11.663607 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a123bb34-21c2-43e4-9182-e8288a2003cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:11.663607 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a123bb34-21c2-43e4-9182-e8288a2003cb None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:11.687175 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:11.688651 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 397/791] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:51:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:51:11.694348 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a123bb34-21c2-43e4-9182-e8288a2003cb None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:11.705976 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73792439-cb58-4bf6-9d21-dddab023e937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:11.706588 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73792439-cb58-4bf6-9d21-dddab023e937 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:11.706865 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73792439-cb58-4bf6-9d21-dddab023e937 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:11.707136 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73792439-cb58-4bf6-9d21-dddab023e937 None admin] PATH_INFO: `/v3/domains` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:11.707779 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73792439-cb58-4bf6-9d21-dddab023e937 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:11.708079 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73792439-cb58-4bf6-9d21-dddab023e937 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:11.709507 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73792439-cb58-4bf6-9d21-dddab023e937 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:11.730160 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 395/792] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:51:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:11.736902 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73792439-cb58-4bf6-9d21-dddab023e937 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:11.864222 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73792439-cb58-4bf6-9d21-dddab023e937 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:11.906266 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796a78aa-5d0b-494b-b232-22732da4b567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:11.906915 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796a78aa-5d0b-494b-b232-22732da4b567 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:11.907192 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796a78aa-5d0b-494b-b232-22732da4b567 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:11.907511 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796a78aa-5d0b-494b-b232-22732da4b567 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:11.915918 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a123bb34-21c2-43e4-9182-e8288a2003cb None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:11.917190 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796a78aa-5d0b-494b-b232-22732da4b567 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:11.919759 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796a78aa-5d0b-494b-b232-22732da4b567 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:11.932487 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-796a78aa-5d0b-494b-b232-22732da4b567 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 06 21:51:11.932487 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:11.932487 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:11.932487 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:11.932487 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:11.932487 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:11.932487 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:11.932487 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:11.932487 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:11.932487 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:11.932487 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:11.932487 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:11.932487 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 21:51:11.932487 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 21:51:11.932487 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 21:51:11.932487 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 21:51:11.933479 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:11.933479 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:11.933479 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 21:51:11.933479 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 21:51:11.933479 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 21:51:11.933479 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 21:51:11.933479 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:11.933479 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:11.933479 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:11.933479 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:11.933479 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:11.933479 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:11.933479 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:11.933479 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:11.933479 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:11.933479 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:11.933479 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:11.933479 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:11.934437 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:11.934437 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:11.934437 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 21:51:11.934437 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 21:51:11.934437 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 21:51:11.934437 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 21:51:11.934437 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 21:51:11.934437 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 21:51:11.934437 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 06 21:51:11.934437 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:11.934437 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 396/793] 10.222.0.18 () {56 vars in 1093 bytes} [Sat Aug 6 21:51:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:11.941272 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796a78aa-5d0b-494b-b232-22732da4b567 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:11.983775 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:11.993941 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8211bf-41ee-4ab6-bdc8-5873a50cb16e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:11.995974 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8211bf-41ee-4ab6-bdc8-5873a50cb16e None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:11.995974 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8211bf-41ee-4ab6-bdc8-5873a50cb16e None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:11.995974 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8211bf-41ee-4ab6-bdc8-5873a50cb16e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:12.012063 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8211bf-41ee-4ab6-bdc8-5873a50cb16e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:12.015014 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8211bf-41ee-4ab6-bdc8-5873a50cb16e None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:12.033007 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-6f8211bf-41ee-4ab6-bdc8-5873a50cb16e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 06 21:51:12.033007 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:12.033007 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:12.033007 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:12.033007 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:12.033007 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:12.033007 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:12.033007 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:12.033007 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:12.033007 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:12.033007 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:12.033007 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:12.033007 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 21:51:12.033007 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 21:51:12.033007 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 21:51:12.033007 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 21:51:12.034029 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:12.034029 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:12.034029 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 21:51:12.034029 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 21:51:12.034029 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 21:51:12.034029 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 21:51:12.034029 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:12.034029 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:12.034029 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:12.034029 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:12.034029 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:12.034029 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:12.034029 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:12.034029 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:12.034029 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:12.034029 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:12.034029 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:12.034029 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:12.034985 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:12.034985 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:12.034985 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 21:51:12.034985 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 21:51:12.034985 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 21:51:12.034985 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 21:51:12.034985 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 21:51:12.034985 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 21:51:12.034985 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 06 21:51:12.034985 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:12.039640 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 398/794] 10.222.0.18 () {56 vars in 1117 bytes} [Sat Aug 6 21:51:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:12.040805 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c722080b-89b7-4875-8cff-064013890efe None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:12.041043 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c722080b-89b7-4875-8cff-064013890efe None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:12.041318 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c722080b-89b7-4875-8cff-064013890efe None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:12.042786 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 399/795] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:51:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:51:12.047562 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c722080b-89b7-4875-8cff-064013890efe None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:12.089135 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796a78aa-5d0b-494b-b232-22732da4b567 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:12.106207 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:12.126374 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eb67f4-4cc7-497e-a6a4-6e9d029fe500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:12.127375 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eb67f4-4cc7-497e-a6a4-6e9d029fe500 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:12.127854 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eb67f4-4cc7-497e-a6a4-6e9d029fe500 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:12.128319 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eb67f4-4cc7-497e-a6a4-6e9d029fe500 None admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:12.129266 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eb67f4-4cc7-497e-a6a4-6e9d029fe500 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:12.129777 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eb67f4-4cc7-497e-a6a4-6e9d029fe500 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:12.131409 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eb67f4-4cc7-497e-a6a4-6e9d029fe500 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:12.152482 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 397/796] 10.222.0.18 () {56 vars in 1133 bytes} [Sat Aug 6 21:51:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:51:12.156640 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9863c0d-50dd-4e4e-99c8-d01788aef701 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:12.157019 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9863c0d-50dd-4e4e-99c8-d01788aef701 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:12.157407 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9863c0d-50dd-4e4e-99c8-d01788aef701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:12.160719 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c722080b-89b7-4875-8cff-064013890efe None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:12.174858 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-d9863c0d-50dd-4e4e-99c8-d01788aef701 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:51:12.176114 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-d9863c0d-50dd-4e4e-99c8-d01788aef701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:51:12.197109 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59707d4-67db-40aa-bca2-82e859c19a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:12.197768 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59707d4-67db-40aa-bca2-82e859c19a0b None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:12.198057 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59707d4-67db-40aa-bca2-82e859c19a0b None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:12.198336 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59707d4-67db-40aa-bca2-82e859c19a0b None admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:12.199069 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59707d4-67db-40aa-bca2-82e859c19a0b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:12.199380 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59707d4-67db-40aa-bca2-82e859c19a0b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:12.201075 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59707d4-67db-40aa-bca2-82e859c19a0b None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:12.220220 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 400/797] 10.222.0.18 () {56 vars in 1149 bytes} [Sat Aug 6 21:51:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:12.221248 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59707d4-67db-40aa-bca2-82e859c19a0b None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:12.245939 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 398/798] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:51:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 21:51:12.246929 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9863c0d-50dd-4e4e-99c8-d01788aef701 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:12.271513 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:12.350132 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9863c0d-50dd-4e4e-99c8-d01788aef701 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:12.365105 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59707d4-67db-40aa-bca2-82e859c19a0b None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:12.378151 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:12.385278 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98034245-5786-4ba0-bfd3-609c4c207c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:12.385934 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98034245-5786-4ba0-bfd3-609c4c207c8e None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:12.386166 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98034245-5786-4ba0-bfd3-609c4c207c8e None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:12.386437 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98034245-5786-4ba0-bfd3-609c4c207c8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:12.398096 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98034245-5786-4ba0-bfd3-609c4c207c8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:12.399825 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98034245-5786-4ba0-bfd3-609c4c207c8e None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:12.407045 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e1222f-4c78-496f-8ffd-5a3ccc1b3aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:12.408008 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e1222f-4c78-496f-8ffd-5a3ccc1b3aee None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:12.408599 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e1222f-4c78-496f-8ffd-5a3ccc1b3aee None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:12.409106 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e1222f-4c78-496f-8ffd-5a3ccc1b3aee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:12.415561 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-98034245-5786-4ba0-bfd3-609c4c207c8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:51:12.415561 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:12.415561 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:12.415561 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:12.415561 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:12.415561 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:12.415561 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:12.415561 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:12.415561 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:12.415561 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:12.415561 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:12.415561 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:12.415561 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:51:12.415561 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:51:12.415561 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:51:12.415561 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:51:12.416557 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:12.416557 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:12.416557 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:12.416557 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:12.416557 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:12.416557 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:12.416557 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:12.416557 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:12.416557 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:12.416557 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:12.416557 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:12.416557 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:12.416557 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:12.416557 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:12.416557 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:12.416557 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:12.416557 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:51:12.417574 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:51:12.417574 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:51:12.417574 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:12.417574 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 399/799] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:51:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:12.418227 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98034245-5786-4ba0-bfd3-609c4c207c8e None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:12.489104 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e1222f-4c78-496f-8ffd-5a3ccc1b3aee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:12.490617 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e1222f-4c78-496f-8ffd-5a3ccc1b3aee None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:12.501917 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:12.528119 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98034245-5786-4ba0-bfd3-609c4c207c8e None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:12.564797 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04f128c-6710-402b-9b85-a3530fcfc613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:12.565429 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04f128c-6710-402b-9b85-a3530fcfc613 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:12.565710 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04f128c-6710-402b-9b85-a3530fcfc613 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:12.565986 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04f128c-6710-402b-9b85-a3530fcfc613 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:12.576480 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04f128c-6710-402b-9b85-a3530fcfc613 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:51:12.581951 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-a04f128c-6710-402b-9b85-a3530fcfc613 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 21:51:12.581951 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:12.581951 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:12.581951 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:12.581951 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:12.581951 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:12.581951 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:12.581951 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:12.581951 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:12.581951 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:12.581951 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:12.581951 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:12.581951 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:51:12.581951 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:51:12.581951 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:51:12.581951 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:51:12.582918 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:51:12.582918 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:51:12.582918 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:51:12.582918 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:51:12.582918 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:12.582918 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:12.582918 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:12.582918 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:12.582918 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:12.582918 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:12.582918 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:12.582918 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:12.582918 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:12.582918 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:12.582918 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:12.582918 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:12.583743 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:12.583743 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:12.583743 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:12.583743 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:12.583743 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:51:12.583743 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:51:12.583743 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:51:12.583743 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:51:12.583743 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:51:12.583743 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:51:12.583743 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 21:51:12.583743 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:12.583743 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 400/800] 10.222.0.18 () {56 vars in 1096 bytes} [Sat Aug 6 21:51:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:12.584445 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683689ba-6a9c-4e04-a63c-4a6d68bed2e1 None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:12.584535 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683689ba-6a9c-4e04-a63c-4a6d68bed2e1 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:12.584941 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683689ba-6a9c-4e04-a63c-4a6d68bed2e1 None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:12.590023 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 401/801] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:51:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:51:12.590789 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683689ba-6a9c-4e04-a63c-4a6d68bed2e1 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:12.601465 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 401/802] 10.222.0.18 () {58 vars in 1282 bytes} [Sat Aug 6 21:51:12 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 21:51:12.602323 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e1222f-4c78-496f-8ffd-5a3ccc1b3aee None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:12.614857 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:12.706699 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e1222f-4c78-496f-8ffd-5a3ccc1b3aee None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:12.707557 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683689ba-6a9c-4e04-a63c-4a6d68bed2e1 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:12.744193 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7ff98a-112a-47b6-8dae-4122e7d5f851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:12.744835 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ff98a-112a-47b6-8dae-4122e7d5f851 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:12.744942 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9d804f-97ee-4b55-9d22-a5b37b96b91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:12.745118 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ff98a-112a-47b6-8dae-4122e7d5f851 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:12.745382 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ff98a-112a-47b6-8dae-4122e7d5f851 None admin] PATH_INFO: `/v3/domains` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:12.745482 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9d804f-97ee-4b55-9d22-a5b37b96b91a None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:12.745749 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9d804f-97ee-4b55-9d22-a5b37b96b91a None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:12.746016 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9d804f-97ee-4b55-9d22-a5b37b96b91a None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:12.746109 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7ff98a-112a-47b6-8dae-4122e7d5f851 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:12.746313 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7ff98a-112a-47b6-8dae-4122e7d5f851 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:12.746664 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9d804f-97ee-4b55-9d22-a5b37b96b91a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:12.746962 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9d804f-97ee-4b55-9d22-a5b37b96b91a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:12.747684 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7ff98a-112a-47b6-8dae-4122e7d5f851 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:12.748309 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9d804f-97ee-4b55-9d22-a5b37b96b91a None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:12.749918 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:12.762407 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 402/803] 10.222.0.18 () {56 vars in 1099 bytes} [Sat Aug 6 21:51:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:12.762407 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee23a86-73c6-44c8-9285-2f7c7e1fdc08 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:12.762407 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee23a86-73c6-44c8-9285-2f7c7e1fdc08 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:12.762407 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee23a86-73c6-44c8-9285-2f7c7e1fdc08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:12.767483 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 402/804] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:51:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:12.768530 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7ff98a-112a-47b6-8dae-4122e7d5f851 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:12.782503 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-9ee23a86-73c6-44c8-9285-2f7c7e1fdc08 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:51:12.783691 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-9ee23a86-73c6-44c8-9285-2f7c7e1fdc08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:51:12.855401 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 403/805] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:51:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 21:51:12.856448 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee23a86-73c6-44c8-9285-2f7c7e1fdc08 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:12.866393 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:12.885012 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7ff98a-112a-47b6-8dae-4122e7d5f851 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:12.919816 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247448d8-5e28-4718-8c63-036f285ff8db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:12.920471 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247448d8-5e28-4718-8c63-036f285ff8db None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:12.920744 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247448d8-5e28-4718-8c63-036f285ff8db None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:12.921634 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247448d8-5e28-4718-8c63-036f285ff8db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:12.931431 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247448d8-5e28-4718-8c63-036f285ff8db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:12.932922 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247448d8-5e28-4718-8c63-036f285ff8db None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:12.947324 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-247448d8-5e28-4718-8c63-036f285ff8db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:51:12.947324 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:12.947324 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:12.947324 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:12.947324 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:12.947324 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:12.947324 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:12.947324 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:12.947324 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:12.947324 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:12.947324 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:12.947324 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:12.947324 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:51:12.947324 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:51:12.947324 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:51:12.947324 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:51:12.948464 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:12.948464 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:12.948464 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:12.948464 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:12.948464 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:12.948464 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:12.948464 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:12.948464 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:12.948464 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:12.948464 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:12.948464 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:12.948464 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:12.948464 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:12.948464 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:12.948464 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:12.948464 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:12.948464 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:51:12.949358 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:51:12.949358 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:51:12.949358 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:12.949358 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 403/806] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:51:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:12.949358 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247448d8-5e28-4718-8c63-036f285ff8db None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:12.969637 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee23a86-73c6-44c8-9285-2f7c7e1fdc08 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:12.994509 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:13.002621 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334596ef-7a84-4879-97ec-26fcf1a2ece3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:13.003242 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334596ef-7a84-4879-97ec-26fcf1a2ece3 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:13.007392 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334596ef-7a84-4879-97ec-26fcf1a2ece3 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:13.007392 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334596ef-7a84-4879-97ec-26fcf1a2ece3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:13.013619 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334596ef-7a84-4879-97ec-26fcf1a2ece3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:13.015719 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334596ef-7a84-4879-97ec-26fcf1a2ece3 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:13.030147 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-334596ef-7a84-4879-97ec-26fcf1a2ece3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:51:13.030147 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:13.030147 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:13.030147 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:13.030147 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:13.030147 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:13.030147 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:13.030147 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:13.030147 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:13.030147 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:13.030147 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:13.030147 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:13.030147 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:51:13.030147 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:51:13.030147 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:51:13.030147 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:51:13.031905 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:13.031905 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:13.031905 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:13.031905 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:13.031905 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:13.031905 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:13.031905 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:13.031905 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:13.031905 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:13.031905 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:13.031905 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:13.031905 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:13.031905 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:13.031905 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:13.031905 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:13.031905 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:13.031905 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:51:13.033168 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:51:13.033168 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:51:13.033168 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:13.033168 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 404/807] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:51:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:13.033168 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334596ef-7a84-4879-97ec-26fcf1a2ece3 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:13.053043 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247448d8-5e28-4718-8c63-036f285ff8db None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:13.086408 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014523e3-d033-4f5e-8921-67906b94d93a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:13.087036 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014523e3-d033-4f5e-8921-67906b94d93a None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:13.087316 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014523e3-d033-4f5e-8921-67906b94d93a None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:13.087587 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014523e3-d033-4f5e-8921-67906b94d93a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:13.087686 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:13.094500 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014523e3-d033-4f5e-8921-67906b94d93a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:13.095996 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014523e3-d033-4f5e-8921-67906b94d93a None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:13.107537 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-014523e3-d033-4f5e-8921-67906b94d93a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 21:51:13.107537 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:13.107537 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:13.107537 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:13.107537 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:13.107537 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:13.107537 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:13.107537 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:13.107537 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:13.107537 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:13.107537 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:13.107537 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:13.107537 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 21:51:13.107537 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 21:51:13.107537 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 21:51:13.109047 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 21:51:13.109047 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:13.109047 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:13.109047 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:13.109047 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:13.109047 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:13.109047 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:13.109047 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:13.109047 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:13.109047 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:13.109047 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:13.109047 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:13.109047 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:13.109047 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:13.109047 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:13.109047 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:13.109047 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:13.109991 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 21:51:13.109991 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 21:51:13.109991 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 21:51:13.109991 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 21:51:13.109991 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 21:51:13.109991 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 21:51:13.109991 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 21:51:13.109991 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:13.109991 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 404/808] 10.222.0.18 () {56 vars in 1111 bytes} [Sat Aug 6 21:51:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:13.110990 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d322ad6-e7bc-490e-9123-8330986051ea None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:13.111337 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d322ad6-e7bc-490e-9123-8330986051ea None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:13.111652 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d322ad6-e7bc-490e-9123-8330986051ea None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:13.113128 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 405/809] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:51:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:51:13.114086 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d322ad6-e7bc-490e-9123-8330986051ea None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:13.137831 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334596ef-7a84-4879-97ec-26fcf1a2ece3 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:13.171984 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5abbcdb-46f2-47b4-9dbf-92eaa2b2fe31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:13.172627 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5abbcdb-46f2-47b4-9dbf-92eaa2b2fe31 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:13.172907 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5abbcdb-46f2-47b4-9dbf-92eaa2b2fe31 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:13.173175 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5abbcdb-46f2-47b4-9dbf-92eaa2b2fe31 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:13.173744 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5abbcdb-46f2-47b4-9dbf-92eaa2b2fe31 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:13.175095 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5abbcdb-46f2-47b4-9dbf-92eaa2b2fe31 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:13.185707 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-d5abbcdb-46f2-47b4-9dbf-92eaa2b2fe31 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 06 21:51:13.185707 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:13.185707 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 21:51:13.185707 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 21:51:13.185707 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 21:51:13.185707 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 21:51:13.185707 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 21:51:13.185707 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 21:51:13.185707 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 06 21:51:13.185707 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:13.185707 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 21:51:13.185707 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:13.185707 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:13.185707 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:13.185707 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:13.185707 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:13.185707 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:13.187002 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:13.187002 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:13.187002 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:13.187002 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:13.187002 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:13.187002 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:13.187002 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 21:51:13.187002 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 21:51:13.187002 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 21:51:13.187002 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 21:51:13.187002 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:13.187002 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:13.187002 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:13.187002 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:13.187002 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:13.187002 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:13.187858 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:13.187858 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:13.187858 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:13.187858 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:13.187858 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:13.187858 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:13.187858 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:13.187858 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:13.187858 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:13.187858 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:13.187858 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 21:51:13.187858 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 21:51:13.187858 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 06 21:51:13.187858 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:13.187858 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 405/810] 10.222.0.18 () {56 vars in 1129 bytes} [Sat Aug 6 21:51:13 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:13.189196 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5abbcdb-46f2-47b4-9dbf-92eaa2b2fe31 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:13.218926 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d322ad6-e7bc-490e-9123-8330986051ea None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:13.226445 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:13.254265 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6f6066-e447-475b-90e0-8e1fad588c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:13.254966 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6f6066-e447-475b-90e0-8e1fad588c91 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:13.255248 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6f6066-e447-475b-90e0-8e1fad588c91 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:13.255522 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6f6066-e447-475b-90e0-8e1fad588c91 None admin] PATH_INFO: `/v3/domains` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:13.256167 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6f6066-e447-475b-90e0-8e1fad588c91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:13.256506 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6f6066-e447-475b-90e0-8e1fad588c91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:13.257877 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6f6066-e447-475b-90e0-8e1fad588c91 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:13.269510 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 406/811] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:51:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:13.270299 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6f6066-e447-475b-90e0-8e1fad588c91 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:13.289850 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5abbcdb-46f2-47b4-9dbf-92eaa2b2fe31 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:13.307262 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:13.324740 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b48bcfc-af23-4d7a-a525-b46f909e6ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:13.325343 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b48bcfc-af23-4d7a-a525-b46f909e6ddf None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:13.325803 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b48bcfc-af23-4d7a-a525-b46f909e6ddf None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:13.326387 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b48bcfc-af23-4d7a-a525-b46f909e6ddf None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:13.327326 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b48bcfc-af23-4d7a-a525-b46f909e6ddf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:13.327687 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b48bcfc-af23-4d7a-a525-b46f909e6ddf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:13.329175 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b48bcfc-af23-4d7a-a525-b46f909e6ddf None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:13.340496 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 406/812] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:51:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:13.341672 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635415e4-03bf-4900-87cd-f3ff7c1b5c47 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:13.341980 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635415e4-03bf-4900-87cd-f3ff7c1b5c47 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:13.342255 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635415e4-03bf-4900-87cd-f3ff7c1b5c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:13.362123 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-635415e4-03bf-4900-87cd-f3ff7c1b5c47 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:51:13.363306 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-635415e4-03bf-4900-87cd-f3ff7c1b5c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:51:13.375501 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6f6066-e447-475b-90e0-8e1fad588c91 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:13.408482 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2282eba4-7c14-4665-b81b-0663aefa6dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:13.409080 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2282eba4-7c14-4665-b81b-0663aefa6dd3 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:13.409362 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2282eba4-7c14-4665-b81b-0663aefa6dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:13.409627 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2282eba4-7c14-4665-b81b-0663aefa6dd3 None admin] PATH_INFO: `/v3/projects` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:13.410320 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2282eba4-7c14-4665-b81b-0663aefa6dd3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:13.410622 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2282eba4-7c14-4665-b81b-0663aefa6dd3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:13.412030 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2282eba4-7c14-4665-b81b-0663aefa6dd3 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:13.423841 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 407/813] 10.222.0.18 () {56 vars in 1148 bytes} [Sat Aug 6 21:51:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:13.424664 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2282eba4-7c14-4665-b81b-0663aefa6dd3 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:13.434642 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 407/814] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:51:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 21:51:13.434642 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635415e4-03bf-4900-87cd-f3ff7c1b5c47 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:13.453078 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:13.524223 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2282eba4-7c14-4665-b81b-0663aefa6dd3 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:13.534847 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635415e4-03bf-4900-87cd-f3ff7c1b5c47 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:13.543813 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:13.557964 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef2f415-2890-4fce-aa69-e6e0d17effd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:13.558619 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef2f415-2890-4fce-aa69-e6e0d17effd6 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:13.558840 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef2f415-2890-4fce-aa69-e6e0d17effd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:13.559113 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef2f415-2890-4fce-aa69-e6e0d17effd6 None admin] PATH_INFO: `/v3/services` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:13.559810 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef2f415-2890-4fce-aa69-e6e0d17effd6 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:13.560114 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef2f415-2890-4fce-aa69-e6e0d17effd6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:13.561517 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef2f415-2890-4fce-aa69-e6e0d17effd6 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:13.567031 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 408/815] 10.222.0.18 () {56 vars in 1124 bytes} [Sat Aug 6 21:51:13 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:13.567826 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef2f415-2890-4fce-aa69-e6e0d17effd6 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:13.568925 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b77cb36-6e85-49ba-99a8-37f43319840b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:13.569551 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b77cb36-6e85-49ba-99a8-37f43319840b None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:13.569827 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b77cb36-6e85-49ba-99a8-37f43319840b None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:13.570096 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b77cb36-6e85-49ba-99a8-37f43319840b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:13.577009 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b77cb36-6e85-49ba-99a8-37f43319840b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:13.578740 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b77cb36-6e85-49ba-99a8-37f43319840b None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:13.590096 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-6b77cb36-6e85-49ba-99a8-37f43319840b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 21:51:13.590096 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:13.590096 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:13.590096 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:13.590096 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:13.590096 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:13.590096 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:13.590096 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:13.590096 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:13.590096 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:13.590096 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:13.590096 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:13.590096 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 21:51:13.590096 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 21:51:13.590096 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 21:51:13.591120 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 21:51:13.591120 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:13.591120 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:13.591120 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:13.591120 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:13.591120 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:13.591120 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:13.591120 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:13.591120 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:13.591120 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:13.591120 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:13.591120 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:13.591120 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:13.591120 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:13.591120 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:13.591120 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:13.591120 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:13.592040 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 21:51:13.592040 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 21:51:13.592040 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 21:51:13.592040 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 21:51:13.592040 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 21:51:13.592040 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 21:51:13.592040 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 21:51:13.592040 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:13.592040 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 408/816] 10.222.0.18 () {56 vars in 1111 bytes} [Sat Aug 6 21:51:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:13.592040 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b77cb36-6e85-49ba-99a8-37f43319840b None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:13.659667 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:13.668580 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef2f415-2890-4fce-aa69-e6e0d17effd6 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:13.693790 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b77cb36-6e85-49ba-99a8-37f43319840b None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:13.701907 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229b98d5-1557-439b-a292-ca32ba9f7972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:13.703114 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b98d5-1557-439b-a292-ca32ba9f7972 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:13.703398 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b98d5-1557-439b-a292-ca32ba9f7972 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:13.703667 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b98d5-1557-439b-a292-ca32ba9f7972 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:13.710521 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229b98d5-1557-439b-a292-ca32ba9f7972 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:13.711951 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229b98d5-1557-439b-a292-ca32ba9f7972 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:13.723146 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-229b98d5-1557-439b-a292-ca32ba9f7972 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 06 21:51:13.723146 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:13.723146 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:13.723146 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:13.723146 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:13.723146 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:13.723146 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:13.723146 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:13.723146 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:13.723146 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:13.723146 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:13.723146 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:13.723146 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 21:51:13.723146 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 21:51:13.723146 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 21:51:13.723146 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 21:51:13.724126 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:13.724126 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:13.724126 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 21:51:13.724126 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 21:51:13.724126 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 21:51:13.724126 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 21:51:13.724126 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:13.724126 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:13.724126 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:13.724126 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:13.724126 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:13.724126 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:13.724126 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:13.724126 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:13.724126 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:13.724126 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:13.724126 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:13.724126 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:13.725132 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:13.725132 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:13.725132 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 21:51:13.725132 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 21:51:13.725132 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 21:51:13.725132 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 21:51:13.725132 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 21:51:13.725132 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 21:51:13.725132 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 06 21:51:13.725132 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:13.725132 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 409/817] 10.222.0.18 () {56 vars in 1096 bytes} [Sat Aug 6 21:51:13 2022] GET /identity/v3/users/swift => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:13.725132 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229b98d5-1557-439b-a292-ca32ba9f7972 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:13.729172 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35ca914-d35c-4970-ae4a-0b0b06884f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:13.729834 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35ca914-d35c-4970-ae4a-0b0b06884f4d None admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:13.730113 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35ca914-d35c-4970-ae4a-0b0b06884f4d None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:13.730384 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35ca914-d35c-4970-ae4a-0b0b06884f4d None admin] PATH_INFO: `/v3/projects/4bb6307eb0644d85a1d1daaa61352697/users/d1b26ba592724dd389584bb800d877a2/roles/d90649d4ef604eeb8edcf0aa22932ca1` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:13.746515 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35ca914-d35c-4970-ae4a-0b0b06884f4d None admin] RBAC: Authorizing `identity:create_grant(project_id=4bb6307eb0644d85a1d1daaa61352697, user_id=d1b26ba592724dd389584bb800d877a2, role_id=d90649d4ef604eeb8edcf0aa22932ca1)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:13.747950 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35ca914-d35c-4970-ae4a-0b0b06884f4d None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:13.811860 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 409/818] 10.222.0.18 () {58 vars in 1439 bytes} [Sat Aug 6 21:51:13 2022] PUT /identity/v3/projects/4bb6307eb0644d85a1d1daaa61352697/users/d1b26ba592724dd389584bb800d877a2/roles/d90649d4ef604eeb8edcf0aa22932ca1 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 21:51:13.812715 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35ca914-d35c-4970-ae4a-0b0b06884f4d None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:13.828044 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229b98d5-1557-439b-a292-ca32ba9f7972 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:13.841590 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:13.867746 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eab518e-7133-42b2-b984-faf0ce1f00ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:13.868431 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eab518e-7133-42b2-b984-faf0ce1f00ab None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:13.868712 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eab518e-7133-42b2-b984-faf0ce1f00ab None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:13.868986 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eab518e-7133-42b2-b984-faf0ce1f00ab None admin] PATH_INFO: `/v3/services/network` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:13.869592 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eab518e-7133-42b2-b984-faf0ce1f00ab None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:13.871032 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eab518e-7133-42b2-b984-faf0ce1f00ab None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:13.880672 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-0eab518e-7133-42b2-b984-faf0ce1f00ab None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 06 21:51:13.880672 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:13.880672 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 21:51:13.880672 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 21:51:13.880672 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 21:51:13.880672 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 21:51:13.880672 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 21:51:13.880672 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 21:51:13.880672 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 06 21:51:13.880672 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:13.880672 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 21:51:13.880672 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:13.880672 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:13.880672 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:13.880672 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:13.880672 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:13.880672 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:13.881875 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:13.881875 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:13.881875 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:13.881875 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:13.881875 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:13.881875 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:13.881875 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 21:51:13.881875 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 21:51:13.881875 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 21:51:13.881875 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 21:51:13.881875 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:13.881875 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:13.881875 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:13.881875 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:13.881875 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:13.881875 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:13.882707 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:13.882707 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:13.882707 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:13.882707 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:13.882707 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:13.882707 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:13.882707 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:13.882707 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:13.882707 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:13.882707 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:13.882707 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 21:51:13.882707 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 21:51:13.882707 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 06 21:51:13.882707 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:13.887083 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 410/819] 10.222.0.18 () {56 vars in 1111 bytes} [Sat Aug 6 21:51:13 2022] GET /identity/v3/services/network => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:13.888060 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eab518e-7133-42b2-b984-faf0ce1f00ab None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:13.906447 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:13.941403 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35ca914-d35c-4970-ae4a-0b0b06884f4d None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:13.985788 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3384740e-6cbb-4c1e-950b-50082cbbf07d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:13.986396 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3384740e-6cbb-4c1e-950b-50082cbbf07d None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:13.986670 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3384740e-6cbb-4c1e-950b-50082cbbf07d None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:13.986936 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3384740e-6cbb-4c1e-950b-50082cbbf07d None admin] PATH_INFO: `/v3/services` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:13.987570 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3384740e-6cbb-4c1e-950b-50082cbbf07d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:13.987847 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3384740e-6cbb-4c1e-950b-50082cbbf07d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:13.989273 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3384740e-6cbb-4c1e-950b-50082cbbf07d None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:13.994435 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 410/820] 10.222.0.18 () {56 vars in 1124 bytes} [Sat Aug 6 21:51:13 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:13.995076 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3384740e-6cbb-4c1e-950b-50082cbbf07d None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:14.006520 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eab518e-7133-42b2-b984-faf0ce1f00ab None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:14.043227 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e758017c-328d-4104-85d5-b487b2c36933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:14.043834 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e758017c-328d-4104-85d5-b487b2c36933 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:14.044111 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e758017c-328d-4104-85d5-b487b2c36933 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:14.044382 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e758017c-328d-4104-85d5-b487b2c36933 None admin] PATH_INFO: `/v3/projects` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:14.045092 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e758017c-328d-4104-85d5-b487b2c36933 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:14.045396 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e758017c-328d-4104-85d5-b487b2c36933 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:14.048035 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e758017c-328d-4104-85d5-b487b2c36933 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:14.060494 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 411/821] 10.222.0.18 () {56 vars in 1148 bytes} [Sat Aug 6 21:51:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:14.065452 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e758017c-328d-4104-85d5-b487b2c36933 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:14.081905 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:14.100876 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3384740e-6cbb-4c1e-950b-50082cbbf07d None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:14.139620 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c83516-bf86-4c8c-ab7a-fbac2b5cb752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:14.140461 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c83516-bf86-4c8c-ab7a-fbac2b5cb752 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:14.140835 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c83516-bf86-4c8c-ab7a-fbac2b5cb752 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:14.141216 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c83516-bf86-4c8c-ab7a-fbac2b5cb752 None admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:14.142249 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c83516-bf86-4c8c-ab7a-fbac2b5cb752 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:14.142661 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c83516-bf86-4c8c-ab7a-fbac2b5cb752 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:14.144166 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c83516-bf86-4c8c-ab7a-fbac2b5cb752 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:14.158751 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:14.163029 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 411/822] 10.222.0.18 () {56 vars in 1135 bytes} [Sat Aug 6 21:51:13 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:14.163673 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c83516-bf86-4c8c-ab7a-fbac2b5cb752 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:14.193268 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e758017c-328d-4104-85d5-b487b2c36933 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:14.228715 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47ae7ae-aca4-4da3-894f-602bebf51601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:14.229327 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47ae7ae-aca4-4da3-894f-602bebf51601 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:14.229601 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47ae7ae-aca4-4da3-894f-602bebf51601 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:14.229960 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47ae7ae-aca4-4da3-894f-602bebf51601 None admin] PATH_INFO: `/v3/services` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:14.230495 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47ae7ae-aca4-4da3-894f-602bebf51601 None admin] RBAC: Adding query filter params (name=network) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:14.230797 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47ae7ae-aca4-4da3-894f-602bebf51601 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:14.232451 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47ae7ae-aca4-4da3-894f-602bebf51601 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:14.241481 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 412/823] 10.222.0.18 () {56 vars in 1112 bytes} [Sat Aug 6 21:51:14 2022] GET /identity/v3/services?name=network => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:14.241481 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47ae7ae-aca4-4da3-894f-602bebf51601 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:14.276984 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c83516-bf86-4c8c-ab7a-fbac2b5cb752 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:14.324337 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df82fa97-1022-4c4c-9381-91dbfdc134af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:14.324901 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df82fa97-1022-4c4c-9381-91dbfdc134af None admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:14.325179 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df82fa97-1022-4c4c-9381-91dbfdc134af None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:14.325451 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df82fa97-1022-4c4c-9381-91dbfdc134af None admin] PATH_INFO: `/v3/endpoints` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:14.326099 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df82fa97-1022-4c4c-9381-91dbfdc134af None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:14.328513 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df82fa97-1022-4c4c-9381-91dbfdc134af None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:14.341352 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:14.353989 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47ae7ae-aca4-4da3-894f-602bebf51601 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:14.388847 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb13d2a-a6e1-405f-b855-d9884687c593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:14.389443 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb13d2a-a6e1-405f-b855-d9884687c593 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:14.390458 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb13d2a-a6e1-405f-b855-d9884687c593 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:14.390847 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb13d2a-a6e1-405f-b855-d9884687c593 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:14.391566 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb13d2a-a6e1-405f-b855-d9884687c593 None admin] RBAC: Adding query filter params (role.id=4b8dcf0491fd4072acc69fd894bf77d0, scope.project.id=4bb6307eb0644d85a1d1daaa61352697, user.id=2a1319d33e9144859c6010843be312a3) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:14.391870 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb13d2a-a6e1-405f-b855-d9884687c593 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:14.393303 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb13d2a-a6e1-405f-b855-d9884687c593 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:14.398876 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 413/824] 10.222.0.18 () {56 vars in 1374 bytes} [Sat Aug 6 21:51:14 2022] GET /identity/v3/role_assignments?user.id=2a1319d33e9144859c6010843be312a3&scope.project.id=4bb6307eb0644d85a1d1daaa61352697&role.id=4b8dcf0491fd4072acc69fd894bf77d0 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:14.399672 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb13d2a-a6e1-405f-b855-d9884687c593 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:14.402077 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:14.467493 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 412/825] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 21:51:14 2022] POST /identity/v3/endpoints => generated 346 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 21:51:14.469763 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df82fa97-1022-4c4c-9381-91dbfdc134af None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:14.537954 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb13d2a-a6e1-405f-b855-d9884687c593 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:14.592235 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9955b90-20f5-43ec-915c-d39828877bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:14.593001 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9955b90-20f5-43ec-915c-d39828877bf3 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:14.593414 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9955b90-20f5-43ec-915c-d39828877bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:14.593694 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9955b90-20f5-43ec-915c-d39828877bf3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:14.609038 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9955b90-20f5-43ec-915c-d39828877bf3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:14.611112 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9955b90-20f5-43ec-915c-d39828877bf3 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:14.623431 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df82fa97-1022-4c4c-9381-91dbfdc134af None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:14.637466 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-c9955b90-20f5-43ec-915c-d39828877bf3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:51:14.637466 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:14.637466 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:14.637466 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:14.637466 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:14.637466 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:14.637466 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:14.637466 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:14.637466 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:14.637466 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:14.637466 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:14.637466 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:14.637466 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:51:14.637466 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:51:14.637466 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:51:14.637466 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:51:14.638703 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:14.638703 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:14.638703 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:14.638703 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:14.638703 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:14.638703 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:14.638703 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:14.638703 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:14.638703 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:14.638703 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:14.638703 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:14.638703 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:14.638703 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:14.638703 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:14.638703 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:14.638703 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:14.638703 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:51:14.639704 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:51:14.639704 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:51:14.639704 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:14.640520 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 414/826] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:51:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:14.643749 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9955b90-20f5-43ec-915c-d39828877bf3 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:14.665020 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245d8ebf-a30f-4456-a7e1-e1640a931a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:14.665642 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245d8ebf-a30f-4456-a7e1-e1640a931a85 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:14.665921 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245d8ebf-a30f-4456-a7e1-e1640a931a85 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:14.666190 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245d8ebf-a30f-4456-a7e1-e1640a931a85 None admin] PATH_INFO: `/v3/services` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:14.667288 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245d8ebf-a30f-4456-a7e1-e1640a931a85 None admin] RBAC: Adding query filter params (type=network) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:14.667588 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245d8ebf-a30f-4456-a7e1-e1640a931a85 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:14.669468 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245d8ebf-a30f-4456-a7e1-e1640a931a85 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:14.677226 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 413/827] 10.222.0.18 () {56 vars in 1112 bytes} [Sat Aug 6 21:51:14 2022] GET /identity/v3/services?type=network => generated 350 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:14.682747 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245d8ebf-a30f-4456-a7e1-e1640a931a85 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:14.719976 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:14.760118 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:14.784426 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9955b90-20f5-43ec-915c-d39828877bf3 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:14.809997 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245d8ebf-a30f-4456-a7e1-e1640a931a85 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:14.825386 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18921b6-5885-4ec5-ab0a-fbe19dc10938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:14.826246 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18921b6-5885-4ec5-ab0a-fbe19dc10938 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:14.826528 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18921b6-5885-4ec5-ab0a-fbe19dc10938 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:14.826804 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18921b6-5885-4ec5-ab0a-fbe19dc10938 None admin] PATH_INFO: `/v3/domains` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:14.828260 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18921b6-5885-4ec5-ab0a-fbe19dc10938 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:14.828596 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18921b6-5885-4ec5-ab0a-fbe19dc10938 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:14.830038 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18921b6-5885-4ec5-ab0a-fbe19dc10938 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:14.846834 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 415/828] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:51:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:14.853407 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18921b6-5885-4ec5-ab0a-fbe19dc10938 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:14.862700 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ab95e9-4225-4067-ae5a-6a537742b386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:14.863322 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ab95e9-4225-4067-ae5a-6a537742b386 None admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:14.863601 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ab95e9-4225-4067-ae5a-6a537742b386 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:14.864243 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ab95e9-4225-4067-ae5a-6a537742b386 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:14.865860 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ab95e9-4225-4067-ae5a-6a537742b386 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:14.869810 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ab95e9-4225-4067-ae5a-6a537742b386 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:14.945387 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 414/829] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 21:51:14 2022] POST /identity/v3/endpoints => generated 337 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 21:51:14.991930 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18921b6-5885-4ec5-ab0a-fbe19dc10938 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:15.005901 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:15.046592 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb7ad05-50b6-4c5e-892d-bd2bbbc19c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:15.047547 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb7ad05-50b6-4c5e-892d-bd2bbbc19c7e None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:15.047833 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb7ad05-50b6-4c5e-892d-bd2bbbc19c7e None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:15.048109 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb7ad05-50b6-4c5e-892d-bd2bbbc19c7e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:15.057810 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb7ad05-50b6-4c5e-892d-bd2bbbc19c7e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:15.059267 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb7ad05-50b6-4c5e-892d-bd2bbbc19c7e None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:15.075088 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-0cb7ad05-50b6-4c5e-892d-bd2bbbc19c7e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 21:51:15.075088 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:15.075088 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:15.075088 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:15.075088 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:15.075088 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:15.075088 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:15.075088 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:15.075088 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:15.075088 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:15.075088 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:15.075088 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:15.075088 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 21:51:15.075088 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 21:51:15.075088 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 21:51:15.076021 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 21:51:15.076021 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:15.076021 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:15.076021 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:15.076021 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:15.076021 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:15.076021 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:15.076021 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:15.076021 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:15.076021 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:15.076021 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:15.076021 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:15.076021 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:15.076021 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:15.076021 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:15.076021 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:15.076021 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:15.078834 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 21:51:15.078834 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 21:51:15.078834 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 21:51:15.078834 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 21:51:15.078834 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 21:51:15.078834 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 21:51:15.078834 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 21:51:15.078834 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:15.078834 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 416/830] 10.222.0.18 () {56 vars in 1111 bytes} [Sat Aug 6 21:51:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:15.082618 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ab95e9-4225-4067-ae5a-6a537742b386 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:15.202905 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ab95e9-4225-4067-ae5a-6a537742b386 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:15.220106 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:15.239850 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666a4184-bda7-447f-9f8f-5139d8b3be89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:15.245223 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666a4184-bda7-447f-9f8f-5139d8b3be89 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:15.245223 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666a4184-bda7-447f-9f8f-5139d8b3be89 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:15.245223 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666a4184-bda7-447f-9f8f-5139d8b3be89 None admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:15.245223 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666a4184-bda7-447f-9f8f-5139d8b3be89 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:15.245223 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666a4184-bda7-447f-9f8f-5139d8b3be89 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:15.245223 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666a4184-bda7-447f-9f8f-5139d8b3be89 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:15.260661 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 415/831] 10.222.0.18 () {56 vars in 1148 bytes} [Sat Aug 6 21:51:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:15.262592 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb7ad05-50b6-4c5e-892d-bd2bbbc19c7e None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:15.383666 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb7ad05-50b6-4c5e-892d-bd2bbbc19c7e None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:15.422345 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c2df42-9426-469b-a680-ec764922d074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:15.423009 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c2df42-9426-469b-a680-ec764922d074 None admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:15.423286 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c2df42-9426-469b-a680-ec764922d074 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:15.423557 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c2df42-9426-469b-a680-ec764922d074 None admin] PATH_INFO: `/v3/projects/4bb6307eb0644d85a1d1daaa61352697/users/e7fc9da7505744608dfd3bbd46a0a9d8/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:15.425498 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c2df42-9426-469b-a680-ec764922d074 None admin] RBAC: Authorizing `identity:create_grant(project_id=4bb6307eb0644d85a1d1daaa61352697, user_id=e7fc9da7505744608dfd3bbd46a0a9d8, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:15.426919 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c2df42-9426-469b-a680-ec764922d074 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:15.526798 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 417/832] 10.222.0.18 () {58 vars in 1439 bytes} [Sat Aug 6 21:51:15 2022] PUT /identity/v3/projects/4bb6307eb0644d85a1d1daaa61352697/users/e7fc9da7505744608dfd3bbd46a0a9d8/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 21:51:16.021901 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9c3ae4-0c61-4f63-9d1e-5073c9ad82c6 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:16.022435 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9c3ae4-0c61-4f63-9d1e-5073c9ad82c6 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:16.022435 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9c3ae4-0c61-4f63-9d1e-5073c9ad82c6 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:16.024040 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 416/833] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:51:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:51:16.029200 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f19702-8db4-4d16-908b-2091fa84f055 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:16.029596 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f19702-8db4-4d16-908b-2091fa84f055 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:16.029754 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f19702-8db4-4d16-908b-2091fa84f055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:16.048465 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-90f19702-8db4-4d16-908b-2091fa84f055 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:51:16.049659 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-90f19702-8db4-4d16-908b-2091fa84f055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:51:16.071480 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:16.121303 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 418/834] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:51:16 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 21:51:16.144638 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9c3ae4-0c61-4f63-9d1e-5073c9ad82c6 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:16.251563 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9c3ae4-0c61-4f63-9d1e-5073c9ad82c6 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:16.287461 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27c9b9b-7ea2-49da-a607-3856938f7976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:16.288015 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27c9b9b-7ea2-49da-a607-3856938f7976 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:16.288295 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27c9b9b-7ea2-49da-a607-3856938f7976 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:16.288619 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27c9b9b-7ea2-49da-a607-3856938f7976 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:16.300029 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27c9b9b-7ea2-49da-a607-3856938f7976 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:51:16.304761 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-c27c9b9b-7ea2-49da-a607-3856938f7976 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 21:51:16.304761 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:16.304761 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:16.304761 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:16.304761 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:16.304761 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:16.304761 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:16.304761 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:16.304761 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:16.304761 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:16.304761 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:16.304761 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:16.304761 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:51:16.304761 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:51:16.304761 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:51:16.304761 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:51:16.305683 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:51:16.305683 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:51:16.305683 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:51:16.305683 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:51:16.305683 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:16.305683 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:16.305683 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:16.305683 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:16.305683 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:16.305683 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:16.305683 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:16.305683 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:16.305683 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:16.305683 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:16.305683 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:16.305683 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:16.306520 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:16.306520 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:16.306520 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:16.306520 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:16.306520 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:51:16.306520 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:51:16.306520 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:51:16.306520 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:51:16.306520 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:51:16.306520 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:51:16.306520 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 21:51:16.306520 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:16.306520 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 417/835] 10.222.0.18 () {56 vars in 1102 bytes} [Sat Aug 6 21:51:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:16.312976 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f19702-8db4-4d16-908b-2091fa84f055 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:16.423999 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f19702-8db4-4d16-908b-2091fa84f055 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:16.461942 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136419cb-591d-48a3-a559-6b0712625074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:16.462553 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136419cb-591d-48a3-a559-6b0712625074 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:16.462836 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136419cb-591d-48a3-a559-6b0712625074 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:16.463111 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136419cb-591d-48a3-a559-6b0712625074 None admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:16.463782 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136419cb-591d-48a3-a559-6b0712625074 None admin] RBAC: Adding query filter params (name=service) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:16.464085 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136419cb-591d-48a3-a559-6b0712625074 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:16.465494 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136419cb-591d-48a3-a559-6b0712625074 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:16.475919 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 419/836] 10.222.0.18 () {56 vars in 1103 bytes} [Sat Aug 6 21:51:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:16.481219 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27c9b9b-7ea2-49da-a607-3856938f7976 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:16.531446 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:16.587569 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27c9b9b-7ea2-49da-a607-3856938f7976 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:16.624537 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b68b615-6f79-4060-bb07-7e20f4f01a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:16.625189 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b68b615-6f79-4060-bb07-7e20f4f01a41 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:16.625475 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b68b615-6f79-4060-bb07-7e20f4f01a41 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:16.625750 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b68b615-6f79-4060-bb07-7e20f4f01a41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:16.636386 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b68b615-6f79-4060-bb07-7e20f4f01a41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:16.638019 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b68b615-6f79-4060-bb07-7e20f4f01a41 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:16.652898 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-6b68b615-6f79-4060-bb07-7e20f4f01a41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:51:16.652898 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:16.652898 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:16.652898 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:16.652898 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:16.652898 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:16.652898 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:16.652898 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:16.652898 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:16.652898 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:16.652898 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:16.652898 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:16.652898 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:51:16.652898 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:51:16.652898 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:51:16.652898 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:51:16.653838 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:16.653838 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:16.653838 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:16.653838 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:16.653838 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:16.653838 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:16.653838 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:16.653838 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:16.653838 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:16.653838 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:16.653838 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:16.653838 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:16.653838 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:16.653838 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:16.653838 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:16.653838 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:16.653838 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:51:16.654713 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:51:16.654713 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:51:16.654713 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:16.654713 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 418/837] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:51:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:16.660459 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136419cb-591d-48a3-a559-6b0712625074 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:16.681286 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:16.732496 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fc6bf9-fbe2-479a-b575-dd2bd67c733f None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:16.733163 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fc6bf9-fbe2-479a-b575-dd2bd67c733f None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:16.734053 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fc6bf9-fbe2-479a-b575-dd2bd67c733f None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:16.738730 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 419/838] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:51:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 21:51:16.741071 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f588e98-c703-4c6d-abbe-6c96f66b45b3 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:16.741586 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f588e98-c703-4c6d-abbe-6c96f66b45b3 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:16.742064 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f588e98-c703-4c6d-abbe-6c96f66b45b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:16.763038 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-5f588e98-c703-4c6d-abbe-6c96f66b45b3 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:51:16.765311 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-5f588e98-c703-4c6d-abbe-6c96f66b45b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:51:16.770143 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136419cb-591d-48a3-a559-6b0712625074 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:16.809536 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f8b323-3810-4276-9496-aa4f30f6d1d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:16.810126 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f8b323-3810-4276-9496-aa4f30f6d1d4 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:16.810400 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f8b323-3810-4276-9496-aa4f30f6d1d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:16.810665 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f8b323-3810-4276-9496-aa4f30f6d1d4 None admin] PATH_INFO: `/v3/domains` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:16.811819 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f8b323-3810-4276-9496-aa4f30f6d1d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:16.812121 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f8b323-3810-4276-9496-aa4f30f6d1d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:16.813532 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f8b323-3810-4276-9496-aa4f30f6d1d4 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:16.833438 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 420/839] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:51:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:16.834544 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f8b323-3810-4276-9496-aa4f30f6d1d4 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:16.852503 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 420/840] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:51:16 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 21:51:16.867919 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f68c0-4a57-4fd3-83a6-f473dd0f913b None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:16.868208 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f68c0-4a57-4fd3-83a6-f473dd0f913b None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:16.868511 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f68c0-4a57-4fd3-83a6-f473dd0f913b None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:16.869952 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 421/841] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:51:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 21:51:16.870517 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957f68c0-4a57-4fd3-83a6-f473dd0f913b None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:16.891596 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:16.924868 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:16.947117 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f8b323-3810-4276-9496-aa4f30f6d1d4 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:16.976964 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957f68c0-4a57-4fd3-83a6-f473dd0f913b None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:16.985200 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68393eca-0e79-432c-b0c9-95b0e73718f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:16.986188 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68393eca-0e79-432c-b0c9-95b0e73718f7 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:16.986608 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68393eca-0e79-432c-b0c9-95b0e73718f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:16.987019 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68393eca-0e79-432c-b0c9-95b0e73718f7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:16.995058 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68393eca-0e79-432c-b0c9-95b0e73718f7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:16.996761 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68393eca-0e79-432c-b0c9-95b0e73718f7 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:17.009749 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-68393eca-0e79-432c-b0c9-95b0e73718f7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 06 21:51:17.009749 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:17.009749 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:17.009749 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:17.009749 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:17.009749 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:17.009749 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:17.009749 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:17.009749 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:17.009749 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:17.009749 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:17.009749 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:17.009749 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 21:51:17.009749 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 21:51:17.009749 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 21:51:17.009749 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 21:51:17.010860 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:17.010860 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:17.010860 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 21:51:17.010860 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 21:51:17.010860 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 21:51:17.010860 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 21:51:17.010860 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:17.010860 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:17.010860 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:17.010860 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:17.010860 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:17.010860 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:17.010860 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:17.010860 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:17.010860 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:17.010860 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:17.010860 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:17.010860 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:17.012118 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:17.012118 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:17.012118 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 21:51:17.012118 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 21:51:17.012118 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 21:51:17.012118 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 21:51:17.012118 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 21:51:17.012118 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 21:51:17.012118 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 06 21:51:17.012118 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:17.012818 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33275acb-a92f-4983-b4d6-830e03463b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:17.012818 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33275acb-a92f-4983-b4d6-830e03463b77 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:17.012995 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33275acb-a92f-4983-b4d6-830e03463b77 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:17.013217 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33275acb-a92f-4983-b4d6-830e03463b77 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:17.013791 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33275acb-a92f-4983-b4d6-830e03463b77 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:17.014549 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 421/842] 10.222.0.18 () {56 vars in 1117 bytes} [Sat Aug 6 21:51:16 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:17.015890 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19750c08-ec59-4b53-9c56-bdcb16f11825 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:17.016329 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19750c08-ec59-4b53-9c56-bdcb16f11825 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:17.016776 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19750c08-ec59-4b53-9c56-bdcb16f11825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:17.019719 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33275acb-a92f-4983-b4d6-830e03463b77 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:17.028538 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-33275acb-a92f-4983-b4d6-830e03463b77 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 06 21:51:17.028538 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:17.028538 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 21:51:17.028538 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 21:51:17.028538 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 21:51:17.028538 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 21:51:17.028538 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 21:51:17.028538 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 21:51:17.028538 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 06 21:51:17.028538 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:17.028538 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 21:51:17.028538 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:17.028538 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:17.028538 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:17.028538 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:17.028538 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:17.029925 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:17.029925 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:17.029925 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:17.029925 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:17.029925 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:17.029925 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:17.029925 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:17.029925 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 21:51:17.029925 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 21:51:17.029925 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 21:51:17.029925 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 21:51:17.029925 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:17.029925 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:17.029925 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:17.029925 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:17.029925 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:17.029925 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:17.031146 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:17.031146 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:17.031146 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:17.031146 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:17.031146 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:17.031146 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:17.031146 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:17.031146 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:17.031146 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:17.031146 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:17.031146 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 21:51:17.031146 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 21:51:17.031146 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 06 21:51:17.031146 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:17.034583 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 422/843] 10.222.0.18 () {56 vars in 1132 bytes} [Sat Aug 6 21:51:16 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:17.035217 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33275acb-a92f-4983-b4d6-830e03463b77 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:17.035615 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-19750c08-ec59-4b53-9c56-bdcb16f11825 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:51:17.036841 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-19750c08-ec59-4b53-9c56-bdcb16f11825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:51:17.115225 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 422/844] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:51:17 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 21:51:17.115225 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19750c08-ec59-4b53-9c56-bdcb16f11825 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:17.130584 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:17.154134 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33275acb-a92f-4983-b4d6-830e03463b77 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:17.187786 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:17.195168 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5418d1a6-d100-4eb8-ad33-ce8a123b4b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:17.195786 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5418d1a6-d100-4eb8-ad33-ce8a123b4b3a None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:17.196062 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5418d1a6-d100-4eb8-ad33-ce8a123b4b3a None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:17.196330 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5418d1a6-d100-4eb8-ad33-ce8a123b4b3a None admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:17.197097 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5418d1a6-d100-4eb8-ad33-ce8a123b4b3a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:17.197402 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5418d1a6-d100-4eb8-ad33-ce8a123b4b3a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:17.198828 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5418d1a6-d100-4eb8-ad33-ce8a123b4b3a None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:17.222788 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 423/845] 10.222.0.18 () {56 vars in 1149 bytes} [Sat Aug 6 21:51:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:17.222788 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5418d1a6-d100-4eb8-ad33-ce8a123b4b3a None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:17.232022 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19750c08-ec59-4b53-9c56-bdcb16f11825 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:17.272182 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52263d13-9abf-40a7-a1c6-0fd3108488aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:17.272824 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52263d13-9abf-40a7-a1c6-0fd3108488aa None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:17.273119 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52263d13-9abf-40a7-a1c6-0fd3108488aa None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:17.273806 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52263d13-9abf-40a7-a1c6-0fd3108488aa None admin] PATH_INFO: `/v3/services` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:17.274450 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52263d13-9abf-40a7-a1c6-0fd3108488aa None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:17.274910 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52263d13-9abf-40a7-a1c6-0fd3108488aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:17.276284 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52263d13-9abf-40a7-a1c6-0fd3108488aa None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:17.281436 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 423/846] 10.222.0.18 () {56 vars in 1126 bytes} [Sat Aug 6 21:51:17 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:17.286123 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52263d13-9abf-40a7-a1c6-0fd3108488aa None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:17.323199 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5418d1a6-d100-4eb8-ad33-ce8a123b4b3a None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:17.369609 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6972680-eea8-4336-8c41-843796ca3a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:17.370169 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6972680-eea8-4336-8c41-843796ca3a48 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:17.370448 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6972680-eea8-4336-8c41-843796ca3a48 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:17.370720 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6972680-eea8-4336-8c41-843796ca3a48 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:17.371300 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6972680-eea8-4336-8c41-843796ca3a48 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:17.372704 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6972680-eea8-4336-8c41-843796ca3a48 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:17.382131 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-a6972680-eea8-4336-8c41-843796ca3a48 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 06 21:51:17.382131 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:17.382131 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 21:51:17.382131 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 21:51:17.382131 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 21:51:17.382131 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 21:51:17.382131 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 21:51:17.382131 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 21:51:17.382131 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 06 21:51:17.382131 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:17.382131 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 21:51:17.382131 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:17.382131 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:17.382131 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:17.382131 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:17.382131 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:17.382131 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:17.383353 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:17.383353 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:17.383353 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:17.383353 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:17.383353 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:17.383353 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:17.383353 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 21:51:17.383353 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 21:51:17.383353 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 21:51:17.383353 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 21:51:17.383353 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:17.383353 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:17.383353 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:17.383353 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:17.383353 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:17.383353 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:17.384186 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:17.384186 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:17.384186 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:17.384186 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:17.384186 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:17.384186 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:17.384186 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:17.384186 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:17.384186 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:17.384186 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:17.384186 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 21:51:17.384186 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 21:51:17.384186 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 06 21:51:17.384186 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:17.384186 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 424/847] 10.222.0.18 () {56 vars in 1114 bytes} [Sat Aug 6 21:51:17 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:17.385030 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6972680-eea8-4336-8c41-843796ca3a48 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:17.392113 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:17.406919 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52263d13-9abf-40a7-a1c6-0fd3108488aa None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:17.440181 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:17.445439 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ad2f07-1d3b-430f-a6b2-6ce0a56c71db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:17.446046 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ad2f07-1d3b-430f-a6b2-6ce0a56c71db None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:17.446326 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ad2f07-1d3b-430f-a6b2-6ce0a56c71db None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:17.446606 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ad2f07-1d3b-430f-a6b2-6ce0a56c71db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:17.500661 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6972680-eea8-4336-8c41-843796ca3a48 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:17.517984 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ad2f07-1d3b-430f-a6b2-6ce0a56c71db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:17.519435 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ad2f07-1d3b-430f-a6b2-6ce0a56c71db None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:17.537482 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03a9db0-40aa-4528-8cac-68f5f7ff29b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:17.538092 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03a9db0-40aa-4528-8cac-68f5f7ff29b6 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:17.538375 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03a9db0-40aa-4528-8cac-68f5f7ff29b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:17.538647 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03a9db0-40aa-4528-8cac-68f5f7ff29b6 None admin] PATH_INFO: `/v3/services` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:17.539277 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03a9db0-40aa-4528-8cac-68f5f7ff29b6 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:17.539580 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03a9db0-40aa-4528-8cac-68f5f7ff29b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:17.541092 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03a9db0-40aa-4528-8cac-68f5f7ff29b6 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:17.552435 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 425/848] 10.222.0.18 () {56 vars in 1126 bytes} [Sat Aug 6 21:51:17 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:17.552435 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03a9db0-40aa-4528-8cac-68f5f7ff29b6 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:17.631178 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:17.637870 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 424/849] 10.222.0.18 () {58 vars in 1282 bytes} [Sat Aug 6 21:51:17 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 21:51:17.644283 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ad2f07-1d3b-430f-a6b2-6ce0a56c71db None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:17.675496 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03a9db0-40aa-4528-8cac-68f5f7ff29b6 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:17.677947 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:17.714461 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d0967f-f947-4163-a347-2e0a4ec9bba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:17.715495 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d0967f-f947-4163-a347-2e0a4ec9bba1 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:17.716343 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d0967f-f947-4163-a347-2e0a4ec9bba1 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:17.717445 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d0967f-f947-4163-a347-2e0a4ec9bba1 None admin] PATH_INFO: `/v3/services` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:17.718357 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d0967f-f947-4163-a347-2e0a4ec9bba1 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:17.719015 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d0967f-f947-4163-a347-2e0a4ec9bba1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:17.720837 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d0967f-f947-4163-a347-2e0a4ec9bba1 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:17.726117 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 426/850] 10.222.0.18 () {56 vars in 1114 bytes} [Sat Aug 6 21:51:17 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:17.728802 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa61adb-4efb-4b24-8e3f-24de52cc84bf None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:17.729273 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa61adb-4efb-4b24-8e3f-24de52cc84bf None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:17.729744 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa61adb-4efb-4b24-8e3f-24de52cc84bf None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:17.731300 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 427/851] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:51:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:51:17.732454 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa61adb-4efb-4b24-8e3f-24de52cc84bf None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:17.755390 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ad2f07-1d3b-430f-a6b2-6ce0a56c71db None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:17.792962 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcf7b43-ec6c-4537-829c-1ec5fa8c2d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:17.793777 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcf7b43-ec6c-4537-829c-1ec5fa8c2d14 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:17.794227 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcf7b43-ec6c-4537-829c-1ec5fa8c2d14 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:17.794502 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcf7b43-ec6c-4537-829c-1ec5fa8c2d14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:17.806112 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcf7b43-ec6c-4537-829c-1ec5fa8c2d14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:17.807694 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcf7b43-ec6c-4537-829c-1ec5fa8c2d14 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:17.824017 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-6bcf7b43-ec6c-4537-829c-1ec5fa8c2d14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:51:17.824017 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:17.824017 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:17.824017 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:17.824017 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:17.824017 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:17.824017 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:17.824017 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:17.824017 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:17.824017 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:17.824017 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:17.824017 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:17.824017 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:51:17.824017 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:51:17.824017 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:51:17.824017 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:51:17.825035 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:17.825035 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:17.825035 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:17.825035 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:17.825035 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:17.825035 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:17.825035 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:17.825035 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:17.825035 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:17.825035 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:17.825035 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:17.825035 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:17.825035 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:17.825035 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:17.825035 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:17.825035 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:17.825035 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:51:17.830923 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:51:17.830923 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:51:17.830923 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:17.830923 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 425/852] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:51:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 21:51:17.830923 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1b91e6-6983-4827-b57f-859f8e6d2cdd None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:17.830923 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1b91e6-6983-4827-b57f-859f8e6d2cdd None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:17.830923 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1b91e6-6983-4827-b57f-859f8e6d2cdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:17.845711 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa61adb-4efb-4b24-8e3f-24de52cc84bf None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:17.846069 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-dc1b91e6-6983-4827-b57f-859f8e6d2cdd None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:51:17.847384 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-dc1b91e6-6983-4827-b57f-859f8e6d2cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:51:17.877152 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:17.882112 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38140d4-5cfa-4e00-95ad-ecc13bf89f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:17.882733 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38140d4-5cfa-4e00-95ad-ecc13bf89f1a None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:17.883077 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38140d4-5cfa-4e00-95ad-ecc13bf89f1a None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:17.883414 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38140d4-5cfa-4e00-95ad-ecc13bf89f1a None admin] PATH_INFO: `/v3/services` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:17.884110 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38140d4-5cfa-4e00-95ad-ecc13bf89f1a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:17.884510 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38140d4-5cfa-4e00-95ad-ecc13bf89f1a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:17.886046 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38140d4-5cfa-4e00-95ad-ecc13bf89f1a None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:17.890759 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 428/853] 10.222.0.18 () {56 vars in 1114 bytes} [Sat Aug 6 21:51:17 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:17.891615 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38140d4-5cfa-4e00-95ad-ecc13bf89f1a None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:17.955132 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 426/854] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:51:17 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 21:51:17.978268 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1b91e6-6983-4827-b57f-859f8e6d2cdd None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:17.999437 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:18.036362 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38140d4-5cfa-4e00-95ad-ecc13bf89f1a None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:18.084812 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bb3627-d013-4ade-bf19-448571c3ef00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:18.085523 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bb3627-d013-4ade-bf19-448571c3ef00 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:18.085872 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bb3627-d013-4ade-bf19-448571c3ef00 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:18.086210 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bb3627-d013-4ade-bf19-448571c3ef00 None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:18.086923 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bb3627-d013-4ade-bf19-448571c3ef00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:18.087225 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bb3627-d013-4ade-bf19-448571c3ef00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:18.088756 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bb3627-d013-4ade-bf19-448571c3ef00 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:18.101521 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 429/855] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:51:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:18.106579 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bb3627-d013-4ade-bf19-448571c3ef00 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:18.120349 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1b91e6-6983-4827-b57f-859f8e6d2cdd None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:18.173316 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dda4c17-92c6-4bab-94d6-55428923200f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:18.174348 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dda4c17-92c6-4bab-94d6-55428923200f None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:18.174834 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dda4c17-92c6-4bab-94d6-55428923200f None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:18.175408 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dda4c17-92c6-4bab-94d6-55428923200f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:18.176808 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:18.192244 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dda4c17-92c6-4bab-94d6-55428923200f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:51:18.198857 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-2dda4c17-92c6-4bab-94d6-55428923200f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 21:51:18.198857 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:18.198857 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:18.198857 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:18.198857 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:18.198857 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:18.198857 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:18.198857 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:18.198857 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:18.198857 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:18.198857 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:18.198857 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:18.198857 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:51:18.198857 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:51:18.198857 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:51:18.198857 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:51:18.200078 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:51:18.200078 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:51:18.200078 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:51:18.200078 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:51:18.200078 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:18.200078 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:18.200078 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:18.200078 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:18.200078 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:18.200078 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:18.200078 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:18.200078 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:18.200078 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:18.200078 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:18.200078 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:18.200078 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:18.200941 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:18.200941 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:18.200941 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:18.200941 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:18.200941 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:51:18.200941 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:51:18.200941 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:51:18.200941 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:51:18.200941 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:51:18.200941 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:51:18.200941 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 21:51:18.200941 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:18.204839 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 427/856] 10.222.0.18 () {56 vars in 1096 bytes} [Sat Aug 6 21:51:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 21:51:18.210221 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dda4c17-92c6-4bab-94d6-55428923200f None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:18.217673 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bb3627-d013-4ade-bf19-448571c3ef00 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:18.255326 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ec51e4-239d-4c5f-8c1a-47093965141a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:18.256341 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec51e4-239d-4c5f-8c1a-47093965141a None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:18.256861 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec51e4-239d-4c5f-8c1a-47093965141a None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:18.257365 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec51e4-239d-4c5f-8c1a-47093965141a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:18.265427 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ec51e4-239d-4c5f-8c1a-47093965141a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:18.267168 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ec51e4-239d-4c5f-8c1a-47093965141a None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:18.278754 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-56ec51e4-239d-4c5f-8c1a-47093965141a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 21:51:18.278754 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:18.278754 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:18.278754 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:18.278754 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:18.278754 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:18.278754 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:18.278754 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:18.278754 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:18.278754 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:18.278754 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:18.278754 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:18.278754 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 21:51:18.278754 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 21:51:18.278754 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 21:51:18.279672 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 21:51:18.279672 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:18.279672 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:18.279672 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:18.279672 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:18.279672 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:18.279672 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:18.279672 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:18.279672 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:18.279672 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:18.279672 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:18.279672 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:18.279672 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:18.279672 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:18.279672 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:18.279672 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:18.279672 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:18.280654 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 21:51:18.280654 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 21:51:18.280654 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 21:51:18.280654 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 21:51:18.280654 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 21:51:18.280654 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 21:51:18.280654 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 21:51:18.280654 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:18.287332 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 430/857] 10.222.0.18 () {56 vars in 1111 bytes} [Sat Aug 6 21:51:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:18.289546 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ec51e4-239d-4c5f-8c1a-47093965141a None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:18.306298 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:18.333715 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dda4c17-92c6-4bab-94d6-55428923200f None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:18.375969 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e80373-3117-4d4a-a0d9-2a416135c579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:18.376622 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e80373-3117-4d4a-a0d9-2a416135c579 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:18.376910 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e80373-3117-4d4a-a0d9-2a416135c579 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:18.377185 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e80373-3117-4d4a-a0d9-2a416135c579 None admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:18.377845 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e80373-3117-4d4a-a0d9-2a416135c579 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:18.378148 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e80373-3117-4d4a-a0d9-2a416135c579 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:18.379520 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e80373-3117-4d4a-a0d9-2a416135c579 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:18.395475 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 428/858] 10.222.0.18 () {56 vars in 1099 bytes} [Sat Aug 6 21:51:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:51:18.396168 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e80373-3117-4d4a-a0d9-2a416135c579 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:18.401691 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ec51e4-239d-4c5f-8c1a-47093965141a None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:18.420209 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:18.439077 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552b5c53-ec38-450c-a074-d110990613e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:18.439996 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552b5c53-ec38-450c-a074-d110990613e8 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:18.440511 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552b5c53-ec38-450c-a074-d110990613e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:18.440978 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552b5c53-ec38-450c-a074-d110990613e8 None admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:18.441853 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552b5c53-ec38-450c-a074-d110990613e8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:18.442361 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552b5c53-ec38-450c-a074-d110990613e8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:18.443870 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552b5c53-ec38-450c-a074-d110990613e8 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:18.457917 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 431/859] 10.222.0.18 () {56 vars in 1148 bytes} [Sat Aug 6 21:51:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:51:18.463648 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552b5c53-ec38-450c-a074-d110990613e8 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:18.538281 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e80373-3117-4d4a-a0d9-2a416135c579 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:18.578314 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60013f2-c14e-4642-82b0-f3bd5ca09246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:18.579118 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60013f2-c14e-4642-82b0-f3bd5ca09246 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:18.579451 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60013f2-c14e-4642-82b0-f3bd5ca09246 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:18.579815 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60013f2-c14e-4642-82b0-f3bd5ca09246 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:18.583083 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:18.587984 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552b5c53-ec38-450c-a074-d110990613e8 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:18.593773 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60013f2-c14e-4642-82b0-f3bd5ca09246 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:18.595338 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60013f2-c14e-4642-82b0-f3bd5ca09246 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:18.610134 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-c60013f2-c14e-4642-82b0-f3bd5ca09246 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:51:18.610134 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:18.610134 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:18.610134 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:18.610134 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:18.610134 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:18.610134 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:18.610134 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:18.610134 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:18.610134 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:18.610134 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:18.610134 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:18.610134 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:51:18.610134 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:51:18.610134 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:51:18.610134 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:51:18.611349 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:18.611349 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:18.611349 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:18.611349 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:18.611349 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:18.611349 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:18.611349 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:18.611349 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:18.611349 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:18.611349 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:18.611349 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:18.611349 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:18.611349 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:18.611349 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:18.611349 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:18.611349 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:18.611349 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:51:18.612467 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:51:18.612467 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:51:18.612467 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:18.618081 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 429/860] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:51:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:18.618783 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60013f2-c14e-4642-82b0-f3bd5ca09246 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:18.638422 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b423874-e17f-42fb-9326-9079c7432edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:18.638938 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b423874-e17f-42fb-9326-9079c7432edc None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:18.639211 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b423874-e17f-42fb-9326-9079c7432edc None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:18.639582 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b423874-e17f-42fb-9326-9079c7432edc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:18.640919 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b423874-e17f-42fb-9326-9079c7432edc None admin] RBAC: Adding query filter params (role.id=d90649d4ef604eeb8edcf0aa22932ca1, scope.project.id=4bb6307eb0644d85a1d1daaa61352697, user.id=d1b26ba592724dd389584bb800d877a2) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:18.641181 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b423874-e17f-42fb-9326-9079c7432edc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:18.642650 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b423874-e17f-42fb-9326-9079c7432edc None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:18.656765 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 432/861] 10.222.0.18 () {56 vars in 1374 bytes} [Sat Aug 6 21:51:18 2022] GET /identity/v3/role_assignments?user.id=d1b26ba592724dd389584bb800d877a2&scope.project.id=4bb6307eb0644d85a1d1daaa61352697&role.id=d90649d4ef604eeb8edcf0aa22932ca1 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:18.742411 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60013f2-c14e-4642-82b0-f3bd5ca09246 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:18.781773 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14833643-3148-425d-8c81-2346a77c0306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:18.782396 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14833643-3148-425d-8c81-2346a77c0306 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:18.782674 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14833643-3148-425d-8c81-2346a77c0306 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:18.782945 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14833643-3148-425d-8c81-2346a77c0306 None admin] PATH_INFO: `/v3/domains` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:18.783613 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14833643-3148-425d-8c81-2346a77c0306 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:18.783917 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14833643-3148-425d-8c81-2346a77c0306 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:18.785557 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14833643-3148-425d-8c81-2346a77c0306 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:18.799597 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 430/862] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:51:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:18.805298 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b423874-e17f-42fb-9326-9079c7432edc None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:18.851414 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:18.915142 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b423874-e17f-42fb-9326-9079c7432edc None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:18.954602 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9410c592-e456-4bb1-bb68-d941aa4ce03f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:18.955241 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9410c592-e456-4bb1-bb68-d941aa4ce03f None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:18.955512 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9410c592-e456-4bb1-bb68-d941aa4ce03f None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:18.955783 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9410c592-e456-4bb1-bb68-d941aa4ce03f None admin] PATH_INFO: `/v3/users/swift` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:18.963841 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9410c592-e456-4bb1-bb68-d941aa4ce03f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:18.965302 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9410c592-e456-4bb1-bb68-d941aa4ce03f None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:18.977128 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-9410c592-e456-4bb1-bb68-d941aa4ce03f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 06 21:51:18.977128 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:18.977128 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:18.977128 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:18.977128 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:18.977128 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:18.977128 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:18.977128 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:18.977128 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:18.977128 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:18.977128 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:18.977128 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:18.977128 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 21:51:18.977128 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 21:51:18.977128 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 21:51:18.977128 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 21:51:18.978102 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:18.978102 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:18.978102 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 21:51:18.978102 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 21:51:18.978102 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 21:51:18.978102 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 21:51:18.978102 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:18.978102 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:18.978102 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:18.978102 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:18.978102 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:18.978102 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:18.978102 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:18.978102 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:18.978102 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:18.978102 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:18.978102 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:18.978102 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:18.979464 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:18.979464 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:18.979464 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 21:51:18.979464 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 21:51:18.979464 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 21:51:18.979464 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 21:51:18.979464 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 21:51:18.979464 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 21:51:18.979464 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 06 21:51:18.979464 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:18.979464 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 433/863] 10.222.0.18 () {56 vars in 1096 bytes} [Sat Aug 6 21:51:18 2022] GET /identity/v3/users/swift => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:18.984507 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14833643-3148-425d-8c81-2346a77c0306 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:19.036014 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:19.092366 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14833643-3148-425d-8c81-2346a77c0306 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:19.133883 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af624fb0-9d6a-47c2-bc1b-6a46e63dd9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:19.134647 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af624fb0-9d6a-47c2-bc1b-6a46e63dd9f0 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:19.134838 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af624fb0-9d6a-47c2-bc1b-6a46e63dd9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:19.135055 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af624fb0-9d6a-47c2-bc1b-6a46e63dd9f0 None admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:19.135833 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af624fb0-9d6a-47c2-bc1b-6a46e63dd9f0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:19.136136 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af624fb0-9d6a-47c2-bc1b-6a46e63dd9f0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:19.137597 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af624fb0-9d6a-47c2-bc1b-6a46e63dd9f0 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:19.155993 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 431/864] 10.222.0.18 () {56 vars in 1135 bytes} [Sat Aug 6 21:51:18 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:19.161339 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9410c592-e456-4bb1-bb68-d941aa4ce03f None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:19.269345 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:19.272583 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9410c592-e456-4bb1-bb68-d941aa4ce03f None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:19.307264 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374b5036-036f-44ff-ab9f-38449a79b9fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:19.307790 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b5036-036f-44ff-ab9f-38449a79b9fb None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:19.308079 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b5036-036f-44ff-ab9f-38449a79b9fb None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:19.308325 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b5036-036f-44ff-ab9f-38449a79b9fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:19.383003 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374b5036-036f-44ff-ab9f-38449a79b9fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:19.384479 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374b5036-036f-44ff-ab9f-38449a79b9fb None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:19.488167 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:19.491706 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 434/865] 10.222.0.18 () {58 vars in 1282 bytes} [Sat Aug 6 21:51:19 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 21:51:19.501545 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af624fb0-9d6a-47c2-bc1b-6a46e63dd9f0 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:19.615887 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af624fb0-9d6a-47c2-bc1b-6a46e63dd9f0 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:19.623729 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:19.653049 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9486b190-5e9c-4525-9a81-78f14e7e8d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:19.653688 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9486b190-5e9c-4525-9a81-78f14e7e8d00 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:19.653967 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9486b190-5e9c-4525-9a81-78f14e7e8d00 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:19.654244 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9486b190-5e9c-4525-9a81-78f14e7e8d00 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:19.665839 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9486b190-5e9c-4525-9a81-78f14e7e8d00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:19.667302 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9486b190-5e9c-4525-9a81-78f14e7e8d00 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:19.683899 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-9486b190-5e9c-4525-9a81-78f14e7e8d00 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:51:19.683899 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:19.683899 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:19.683899 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:19.683899 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:19.683899 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:19.683899 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:19.683899 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:19.683899 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:19.683899 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:19.683899 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:19.683899 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:19.683899 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:51:19.683899 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:51:19.683899 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:51:19.683899 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:51:19.686919 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:19.686919 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:19.686919 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:19.686919 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:19.686919 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:19.686919 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:19.686919 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:19.686919 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:19.686919 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:19.686919 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:19.686919 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:19.686919 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:19.686919 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:19.686919 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:19.686919 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:19.686919 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:19.686919 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:51:19.688907 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:51:19.688907 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:51:19.688907 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:19.688907 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 432/866] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:51:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:19.693655 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374b5036-036f-44ff-ab9f-38449a79b9fb None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:19.787944 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1772e590-91f1-47c9-be0c-6308cf69c815 None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:19.788292 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1772e590-91f1-47c9-be0c-6308cf69c815 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:19.788519 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1772e590-91f1-47c9-be0c-6308cf69c815 None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:19.790026 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 433/867] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:51:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:51:19.795430 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1124d22a-29c4-44dd-ae88-014edb47f991 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:19.796250 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1124d22a-29c4-44dd-ae88-014edb47f991 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:19.796553 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1124d22a-29c4-44dd-ae88-014edb47f991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:19.806920 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374b5036-036f-44ff-ab9f-38449a79b9fb None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:19.821051 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-1124d22a-29c4-44dd-ae88-014edb47f991 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:51:19.822473 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-1124d22a-29c4-44dd-ae88-014edb47f991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:51:19.854383 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e014cec-eb41-4f0e-b69a-a3f52417702a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:19.855423 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e014cec-eb41-4f0e-b69a-a3f52417702a None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:19.855919 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e014cec-eb41-4f0e-b69a-a3f52417702a None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:19.856381 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e014cec-eb41-4f0e-b69a-a3f52417702a None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:19.857533 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e014cec-eb41-4f0e-b69a-a3f52417702a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:19.858050 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e014cec-eb41-4f0e-b69a-a3f52417702a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:19.860146 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e014cec-eb41-4f0e-b69a-a3f52417702a None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:19.873168 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 435/868] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:51:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:19.880104 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e014cec-eb41-4f0e-b69a-a3f52417702a None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:19.920428 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 434/869] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:51:19 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 21:51:19.944686 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1124d22a-29c4-44dd-ae88-014edb47f991 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:19.946344 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:20.004900 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e014cec-eb41-4f0e-b69a-a3f52417702a None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:20.050087 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2e07a1-ccf9-4d01-aac5-eada5684c2a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:20.051740 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2e07a1-ccf9-4d01-aac5-eada5684c2a5 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:20.052254 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2e07a1-ccf9-4d01-aac5-eada5684c2a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:20.052754 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2e07a1-ccf9-4d01-aac5-eada5684c2a5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:20.063487 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2e07a1-ccf9-4d01-aac5-eada5684c2a5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:20.066026 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2e07a1-ccf9-4d01-aac5-eada5684c2a5 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:20.068959 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:20.082489 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-ea2e07a1-ccf9-4d01-aac5-eada5684c2a5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 21:51:20.082489 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:20.082489 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:20.082489 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:20.082489 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:20.082489 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:20.082489 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:20.082489 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:20.082489 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:20.082489 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:20.082489 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:20.082489 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:20.082489 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 21:51:20.082489 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 21:51:20.082489 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 21:51:20.083427 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 21:51:20.083427 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:20.083427 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:20.083427 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:20.083427 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:20.083427 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:20.083427 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:20.083427 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:20.083427 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:20.083427 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:20.083427 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:20.083427 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:20.083427 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:20.083427 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:20.083427 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:20.083427 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:20.083427 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:20.084324 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 21:51:20.084324 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 21:51:20.084324 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 21:51:20.084324 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 21:51:20.084324 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 21:51:20.084324 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 21:51:20.084324 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 21:51:20.084324 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:20.090729 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 436/870] 10.222.0.18 () {56 vars in 1111 bytes} [Sat Aug 6 21:51:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 21:51:20.091502 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2e07a1-ccf9-4d01-aac5-eada5684c2a5 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:20.096366 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1124d22a-29c4-44dd-ae88-014edb47f991 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:20.140689 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63567bd4-eca8-4cb4-9973-22e8c9358c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:20.141314 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63567bd4-eca8-4cb4-9973-22e8c9358c7b None admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:20.141630 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63567bd4-eca8-4cb4-9973-22e8c9358c7b None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:20.141910 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63567bd4-eca8-4cb4-9973-22e8c9358c7b None admin] PATH_INFO: `/v3/services` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:20.142567 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63567bd4-eca8-4cb4-9973-22e8c9358c7b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:20.146103 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63567bd4-eca8-4cb4-9973-22e8c9358c7b None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:20.205644 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2e07a1-ccf9-4d01-aac5-eada5684c2a5 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:20.224976 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 435/871] 10.222.0.18 () {60 vars in 1141 bytes} [Sat Aug 6 21:51:19 2022] POST /identity/v3/services => generated 269 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 21:51:20.226239 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977cd585-8fcc-4246-99a6-0849789cc837 None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:20.226528 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977cd585-8fcc-4246-99a6-0849789cc837 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:20.226801 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977cd585-8fcc-4246-99a6-0849789cc837 None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:20.228217 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 436/872] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:51:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:51:20.235076 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b646419-8e22-4627-923d-c31881d7d920 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:20.235439 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b646419-8e22-4627-923d-c31881d7d920 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:20.235628 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b646419-8e22-4627-923d-c31881d7d920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:20.242159 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:20.258736 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-6b646419-8e22-4627-923d-c31881d7d920 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:51:20.259919 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-6b646419-8e22-4627-923d-c31881d7d920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:51:20.271075 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143c7f59-03ef-46c9-bda3-79195ec8adee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:20.272254 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c7f59-03ef-46c9-bda3-79195ec8adee None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:20.272254 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c7f59-03ef-46c9-bda3-79195ec8adee None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:20.272254 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c7f59-03ef-46c9-bda3-79195ec8adee None admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:20.273015 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143c7f59-03ef-46c9-bda3-79195ec8adee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:20.273393 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143c7f59-03ef-46c9-bda3-79195ec8adee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:20.275011 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143c7f59-03ef-46c9-bda3-79195ec8adee None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:20.292256 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 437/873] 10.222.0.18 () {56 vars in 1148 bytes} [Sat Aug 6 21:51:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:51:20.301220 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143c7f59-03ef-46c9-bda3-79195ec8adee None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:20.368526 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 437/874] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:51:20 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 21:51:20.392967 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b646419-8e22-4627-923d-c31881d7d920 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:20.458629 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143c7f59-03ef-46c9-bda3-79195ec8adee None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:20.487369 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:20.508063 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3018f5-7620-421a-8ffe-e0920b249b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:20.509051 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3018f5-7620-421a-8ffe-e0920b249b1b None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:20.509317 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b646419-8e22-4627-923d-c31881d7d920 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:20.509768 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3018f5-7620-421a-8ffe-e0920b249b1b None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:20.510237 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3018f5-7620-421a-8ffe-e0920b249b1b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:20.511330 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3018f5-7620-421a-8ffe-e0920b249b1b None admin] RBAC: Adding query filter params (role.id=4b8dcf0491fd4072acc69fd894bf77d0, scope.project.id=4bb6307eb0644d85a1d1daaa61352697, user.id=e7fc9da7505744608dfd3bbd46a0a9d8) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:20.512710 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3018f5-7620-421a-8ffe-e0920b249b1b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:20.514899 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3018f5-7620-421a-8ffe-e0920b249b1b None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:20.524287 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 438/875] 10.222.0.18 () {56 vars in 1374 bytes} [Sat Aug 6 21:51:20 2022] GET /identity/v3/role_assignments?user.id=e7fc9da7505744608dfd3bbd46a0a9d8&scope.project.id=4bb6307eb0644d85a1d1daaa61352697&role.id=4b8dcf0491fd4072acc69fd894bf77d0 => generated 608 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:20.557141 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b44a278-e4b7-4f42-91ce-0e2bdc1c9ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:20.557776 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b44a278-e4b7-4f42-91ce-0e2bdc1c9ec7 None admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:20.558052 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b44a278-e4b7-4f42-91ce-0e2bdc1c9ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:20.558322 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b44a278-e4b7-4f42-91ce-0e2bdc1c9ec7 None admin] PATH_INFO: `/v3/services` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:20.559005 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b44a278-e4b7-4f42-91ce-0e2bdc1c9ec7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:20.560440 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b44a278-e4b7-4f42-91ce-0e2bdc1c9ec7 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:20.667064 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 438/876] 10.222.0.18 () {60 vars in 1141 bytes} [Sat Aug 6 21:51:20 2022] POST /identity/v3/services => generated 251 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 21:51:20.955288 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f969c60-ce67-4d94-8383-b8c766027b6b None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:20.955658 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f969c60-ce67-4d94-8383-b8c766027b6b None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:20.955854 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f969c60-ce67-4d94-8383-b8c766027b6b None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:20.957544 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 439/877] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:51:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:51:20.963008 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c974141-4bfb-4ff5-8618-53d50b682ca6 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:20.963300 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c974141-4bfb-4ff5-8618-53d50b682ca6 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:20.963578 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c974141-4bfb-4ff5-8618-53d50b682ca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:20.982558 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-5c974141-4bfb-4ff5-8618-53d50b682ca6 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:51:20.983817 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-5c974141-4bfb-4ff5-8618-53d50b682ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:51:21.059697 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 439/878] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:51:20 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 21:51:21.083403 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f969c60-ce67-4d94-8383-b8c766027b6b None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:21.131919 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:21.193119 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f969c60-ce67-4d94-8383-b8c766027b6b None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:21.229494 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f617a5f6-4ba6-49e7-88ae-c2197fd819d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:21.230911 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f617a5f6-4ba6-49e7-88ae-c2197fd819d0 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:21.231391 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f617a5f6-4ba6-49e7-88ae-c2197fd819d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:21.231879 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f617a5f6-4ba6-49e7-88ae-c2197fd819d0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:21.243442 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f617a5f6-4ba6-49e7-88ae-c2197fd819d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:51:21.248318 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-f617a5f6-4ba6-49e7-88ae-c2197fd819d0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 06 21:51:21.248318 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:21.248318 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:21.248318 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:21.248318 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:21.248318 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:21.248318 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:21.248318 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:21.248318 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:21.248318 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:21.248318 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:21.248318 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:21.248318 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:51:21.248318 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:51:21.248318 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:51:21.248318 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:51:21.249412 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:51:21.249412 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:51:21.249412 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:51:21.249412 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:51:21.249412 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:21.249412 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:21.249412 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:21.249412 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:21.249412 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:21.249412 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:21.249412 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:21.249412 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:21.249412 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:21.249412 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:21.249412 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:21.249412 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:21.250410 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:21.250410 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:21.250410 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:21.250410 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:21.250410 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:51:21.250410 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:51:21.250410 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:51:21.250410 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:51:21.250410 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:51:21.250410 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:51:21.250410 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 06 21:51:21.250410 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:21.253037 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 440/879] 10.222.0.18 () {56 vars in 1120 bytes} [Sat Aug 6 21:51:21 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:21.258918 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c974141-4bfb-4ff5-8618-53d50b682ca6 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:21.321613 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:21.361440 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c974141-4bfb-4ff5-8618-53d50b682ca6 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:21.404199 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25cae5d-ec88-4121-b765-2048a7eb29c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:21.405260 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25cae5d-ec88-4121-b765-2048a7eb29c3 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:21.405755 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25cae5d-ec88-4121-b765-2048a7eb29c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:21.406045 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25cae5d-ec88-4121-b765-2048a7eb29c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:21.406707 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25cae5d-ec88-4121-b765-2048a7eb29c3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:21.407006 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25cae5d-ec88-4121-b765-2048a7eb29c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:21.408355 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25cae5d-ec88-4121-b765-2048a7eb29c3 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:21.418941 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 440/880] 10.222.0.18 () {56 vars in 1115 bytes} [Sat Aug 6 21:51:21 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:21.425184 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f617a5f6-4ba6-49e7-88ae-c2197fd819d0 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:21.530662 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:21.533572 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f617a5f6-4ba6-49e7-88ae-c2197fd819d0 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:21.568515 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a57560-e914-4081-b29a-240a31e53be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:21.569145 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a57560-e914-4081-b29a-240a31e53be6 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:21.569426 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a57560-e914-4081-b29a-240a31e53be6 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:21.569698 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a57560-e914-4081-b29a-240a31e53be6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:21.579995 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a57560-e914-4081-b29a-240a31e53be6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:21.581747 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a57560-e914-4081-b29a-240a31e53be6 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:21.597296 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-53a57560-e914-4081-b29a-240a31e53be6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:51:21.597296 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:21.597296 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:21.597296 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:21.597296 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:21.597296 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:21.597296 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:21.597296 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:21.597296 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:21.597296 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:21.597296 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:21.597296 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:21.597296 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:51:21.597296 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:51:21.597296 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:51:21.597296 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:51:21.598235 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:21.598235 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:21.598235 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:21.598235 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:21.598235 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:21.598235 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:21.598235 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:21.598235 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:21.598235 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:21.598235 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:21.598235 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:21.598235 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:21.598235 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:21.598235 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:21.598235 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:21.598235 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:21.598235 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:51:21.599130 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:51:21.599130 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:51:21.599130 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:21.599130 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 441/881] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:51:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:21.605441 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25cae5d-ec88-4121-b765-2048a7eb29c3 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:21.709057 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25cae5d-ec88-4121-b765-2048a7eb29c3 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:21.736861 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:21.746341 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf57138-0b86-44d8-aca5-fa6c78fffa4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:21.747268 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf57138-0b86-44d8-aca5-fa6c78fffa4e None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:21.747707 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf57138-0b86-44d8-aca5-fa6c78fffa4e None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:21.748130 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf57138-0b86-44d8-aca5-fa6c78fffa4e None admin] PATH_INFO: `/v3/domains` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:21.752255 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf57138-0b86-44d8-aca5-fa6c78fffa4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:21.752255 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf57138-0b86-44d8-aca5-fa6c78fffa4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:21.752255 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf57138-0b86-44d8-aca5-fa6c78fffa4e None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:21.766639 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 441/882] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:51:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:21.773644 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a57560-e914-4081-b29a-240a31e53be6 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:21.885938 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a57560-e914-4081-b29a-240a31e53be6 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:21.920432 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca54e86f-db80-43b9-aa7e-7cd4312bbbb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:21.921633 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca54e86f-db80-43b9-aa7e-7cd4312bbbb8 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:21.921906 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca54e86f-db80-43b9-aa7e-7cd4312bbbb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:21.922173 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca54e86f-db80-43b9-aa7e-7cd4312bbbb8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:21.929716 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca54e86f-db80-43b9-aa7e-7cd4312bbbb8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:21.931155 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca54e86f-db80-43b9-aa7e-7cd4312bbbb8 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:21.942725 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-ca54e86f-db80-43b9-aa7e-7cd4312bbbb8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 06 21:51:21.942725 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:21.942725 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:21.942725 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:21.942725 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:21.942725 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:21.942725 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:21.942725 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:21.942725 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:21.942725 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:21.942725 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:21.942725 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:21.942725 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 21:51:21.942725 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 21:51:21.942725 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 21:51:21.942725 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 21:51:21.943673 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:21.943673 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:21.943673 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 21:51:21.943673 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 21:51:21.943673 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 21:51:21.943673 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 21:51:21.943673 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:21.943673 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:21.943673 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:21.943673 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:21.943673 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:21.943673 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:21.943673 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:21.943673 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:21.943673 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:21.943673 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:21.943673 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:21.943673 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:21.944648 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:21.944648 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:21.944648 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 21:51:21.944648 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 21:51:21.944648 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 21:51:21.944648 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 21:51:21.944648 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 21:51:21.944648 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 21:51:21.944648 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 06 21:51:21.944648 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:21.944648 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 442/883] 10.222.0.18 () {56 vars in 1117 bytes} [Sat Aug 6 21:51:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:21.951758 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf57138-0b86-44d8-aca5-fa6c78fffa4e None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:22.053616 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf57138-0b86-44d8-aca5-fa6c78fffa4e None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:22.088151 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d925f324-6546-4bf4-98c5-4a9442be6609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:22.088777 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d925f324-6546-4bf4-98c5-4a9442be6609 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:22.089058 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d925f324-6546-4bf4-98c5-4a9442be6609 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:22.089319 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d925f324-6546-4bf4-98c5-4a9442be6609 None admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:22.090052 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d925f324-6546-4bf4-98c5-4a9442be6609 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:22.090359 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d925f324-6546-4bf4-98c5-4a9442be6609 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:22.091795 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d925f324-6546-4bf4-98c5-4a9442be6609 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:22.111087 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 442/884] 10.222.0.18 () {56 vars in 1149 bytes} [Sat Aug 6 21:51:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:22.117500 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca54e86f-db80-43b9-aa7e-7cd4312bbbb8 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:22.136844 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:22.230884 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca54e86f-db80-43b9-aa7e-7cd4312bbbb8 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:22.267240 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfee7cc-d44a-4fd5-b49a-c362694bce71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:22.267847 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfee7cc-d44a-4fd5-b49a-c362694bce71 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:22.268123 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfee7cc-d44a-4fd5-b49a-c362694bce71 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:22.268393 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfee7cc-d44a-4fd5-b49a-c362694bce71 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:22.337465 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfee7cc-d44a-4fd5-b49a-c362694bce71 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:22.338917 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfee7cc-d44a-4fd5-b49a-c362694bce71 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:22.372808 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:22.455097 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 443/885] 10.222.0.18 () {58 vars in 1282 bytes} [Sat Aug 6 21:51:22 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 21:51:22.460691 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d925f324-6546-4bf4-98c5-4a9442be6609 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:22.488859 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07c4a4c-3383-4731-a7ea-77590f64d4d3 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:22.489175 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07c4a4c-3383-4731-a7ea-77590f64d4d3 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:22.489411 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07c4a4c-3383-4731-a7ea-77590f64d4d3 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:22.490993 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 444/886] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:51:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 21:51:22.495909 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec6dc7e-8945-41b5-9505-7a8d4187c824 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:22.496188 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec6dc7e-8945-41b5-9505-7a8d4187c824 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:22.496490 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec6dc7e-8945-41b5-9505-7a8d4187c824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:22.515657 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-4ec6dc7e-8945-41b5-9505-7a8d4187c824 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:51:22.515976 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:22.516887 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-4ec6dc7e-8945-41b5-9505-7a8d4187c824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:51:22.571327 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d925f324-6546-4bf4-98c5-4a9442be6609 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:22.593103 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 445/887] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:51:22 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 21:51:22.606966 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a87d435-b71c-4c01-b9e4-b6525ad7558b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:22.608066 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a87d435-b71c-4c01-b9e4-b6525ad7558b None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:22.608531 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a87d435-b71c-4c01-b9e4-b6525ad7558b None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:22.609306 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a87d435-b71c-4c01-b9e4-b6525ad7558b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:22.610234 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec6dc7e-8945-41b5-9505-7a8d4187c824 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:22.623078 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a87d435-b71c-4c01-b9e4-b6525ad7558b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:22.626136 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a87d435-b71c-4c01-b9e4-b6525ad7558b None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:22.645560 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-3a87d435-b71c-4c01-b9e4-b6525ad7558b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:51:22.645560 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:22.645560 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:22.645560 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:22.645560 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:22.645560 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:22.645560 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:22.645560 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:22.645560 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:22.645560 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:22.645560 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:22.645560 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:22.645560 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:51:22.645560 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:51:22.645560 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:51:22.645560 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:51:22.646565 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:22.646565 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:22.646565 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:22.646565 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:22.646565 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:22.646565 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:22.646565 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:22.646565 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:22.646565 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:22.646565 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:22.646565 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:22.646565 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:22.646565 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:22.646565 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:22.646565 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:22.646565 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:22.646565 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:51:22.647459 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:51:22.647459 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:51:22.647459 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:22.647459 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 443/888] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:51:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:22.653348 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a87d435-b71c-4c01-b9e4-b6525ad7558b None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:22.658175 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:22.719424 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec6dc7e-8945-41b5-9505-7a8d4187c824 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:22.756320 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25105d3d-721f-465f-9858-7d7f7d060b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:22.756995 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25105d3d-721f-465f-9858-7d7f7d060b29 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:22.757278 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25105d3d-721f-465f-9858-7d7f7d060b29 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:22.757548 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25105d3d-721f-465f-9858-7d7f7d060b29 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:22.758135 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25105d3d-721f-465f-9858-7d7f7d060b29 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:22.758506 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:22.759946 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25105d3d-721f-465f-9858-7d7f7d060b29 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:22.764737 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a87d435-b71c-4c01-b9e4-b6525ad7558b None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:22.770497 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-25105d3d-721f-465f-9858-7d7f7d060b29 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 06 21:51:22.770497 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:22.770497 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 21:51:22.770497 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 21:51:22.770497 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 21:51:22.770497 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 21:51:22.770497 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 21:51:22.770497 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 21:51:22.770497 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 06 21:51:22.770497 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:22.770497 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 21:51:22.770497 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:22.770497 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:22.770497 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:22.770497 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:22.770497 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:22.771419 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:22.771419 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:22.771419 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:22.771419 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:22.771419 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:22.771419 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:22.771419 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:22.771419 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 21:51:22.771419 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 21:51:22.771419 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 21:51:22.771419 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 21:51:22.771419 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:22.771419 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:22.771419 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:22.771419 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:22.771419 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:22.771419 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:22.772347 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:22.772347 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:22.772347 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:22.772347 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:22.772347 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:22.772347 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:22.772347 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:22.772347 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:22.772347 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:22.772347 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:22.772347 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 21:51:22.772347 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 21:51:22.772347 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 06 21:51:22.772347 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:22.772347 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 446/889] 10.222.0.18 () {56 vars in 1132 bytes} [Sat Aug 6 21:51:22 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 21:51:22.773214 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7253c7-ad62-42ec-8472-e402f443efd0 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:22.773214 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7253c7-ad62-42ec-8472-e402f443efd0 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:22.773214 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7253c7-ad62-42ec-8472-e402f443efd0 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:22.773408 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 447/890] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:51:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:51:22.774468 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dfdbb1-ac18-46a6-86c8-dcb1373bc524 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:22.774651 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dfdbb1-ac18-46a6-86c8-dcb1373bc524 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:22.775029 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dfdbb1-ac18-46a6-86c8-dcb1373bc524 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:22.776242 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 448/891] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:51:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:51:22.776943 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dfdbb1-ac18-46a6-86c8-dcb1373bc524 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:22.805158 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4762d623-62b6-4bbc-b45f-1874419f63fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:22.805998 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4762d623-62b6-4bbc-b45f-1874419f63fb None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:22.806276 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4762d623-62b6-4bbc-b45f-1874419f63fb None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:22.806545 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4762d623-62b6-4bbc-b45f-1874419f63fb None admin] PATH_INFO: `/v3/domains` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:22.807187 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4762d623-62b6-4bbc-b45f-1874419f63fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:22.807486 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4762d623-62b6-4bbc-b45f-1874419f63fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:22.808876 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4762d623-62b6-4bbc-b45f-1874419f63fb None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:22.820663 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 444/892] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:51:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:51:22.821819 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deefc93-c600-4cf6-836e-a4368577bc76 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:22.822105 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deefc93-c600-4cf6-836e-a4368577bc76 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:22.822380 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deefc93-c600-4cf6-836e-a4368577bc76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:22.839693 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-6deefc93-c600-4cf6-836e-a4368577bc76 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:51:22.840901 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-6deefc93-c600-4cf6-836e-a4368577bc76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:51:22.880585 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dfdbb1-ac18-46a6-86c8-dcb1373bc524 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:22.887309 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:22.906625 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 445/893] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:51:22 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 21:51:22.907980 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bde344d-43eb-4bbe-9256-7cb12941d010 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:22.908261 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bde344d-43eb-4bbe-9256-7cb12941d010 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:22.908640 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bde344d-43eb-4bbe-9256-7cb12941d010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:22.913984 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a66ece-6646-4762-b5c4-5790ec6d062a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:22.914756 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a66ece-6646-4762-b5c4-5790ec6d062a None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:22.915037 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a66ece-6646-4762-b5c4-5790ec6d062a None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:22.915312 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a66ece-6646-4762-b5c4-5790ec6d062a None admin] PATH_INFO: `/v3/services` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:22.915942 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a66ece-6646-4762-b5c4-5790ec6d062a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:22.916244 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a66ece-6646-4762-b5c4-5790ec6d062a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:22.917658 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a66ece-6646-4762-b5c4-5790ec6d062a None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:22.923564 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 449/894] 10.222.0.18 () {56 vars in 1126 bytes} [Sat Aug 6 21:51:22 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:22.924354 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a66ece-6646-4762-b5c4-5790ec6d062a None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:22.932378 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-6bde344d-43eb-4bbe-9256-7cb12941d010 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:51:22.933578 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-6bde344d-43eb-4bbe-9256-7cb12941d010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:51:23.000226 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 446/895] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:51:22 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 21:51:23.001020 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bde344d-43eb-4bbe-9256-7cb12941d010 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:23.018234 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:23.030092 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a66ece-6646-4762-b5c4-5790ec6d062a None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:23.064634 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44a6a1e-fc70-4ab7-a56d-f89f0bbe0633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:23.065254 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44a6a1e-fc70-4ab7-a56d-f89f0bbe0633 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:23.065531 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44a6a1e-fc70-4ab7-a56d-f89f0bbe0633 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:23.065800 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44a6a1e-fc70-4ab7-a56d-f89f0bbe0633 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:23.072340 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44a6a1e-fc70-4ab7-a56d-f89f0bbe0633 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:23.073807 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44a6a1e-fc70-4ab7-a56d-f89f0bbe0633 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:23.084772 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-e44a6a1e-fc70-4ab7-a56d-f89f0bbe0633 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 21:51:23.084772 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:23.084772 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:23.084772 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:23.084772 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:23.084772 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:23.084772 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:23.084772 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:23.084772 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:23.084772 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:23.084772 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:23.084772 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:23.084772 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 21:51:23.084772 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 21:51:23.084772 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 21:51:23.085807 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 21:51:23.085807 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:23.085807 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:23.085807 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:23.085807 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:23.085807 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:23.085807 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:23.085807 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:23.085807 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:23.085807 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:23.085807 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:23.085807 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:23.085807 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:23.085807 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:23.085807 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:23.085807 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:23.085807 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:23.086822 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 21:51:23.086822 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 21:51:23.086822 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 21:51:23.086822 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 21:51:23.086822 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 21:51:23.086822 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 21:51:23.086822 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 21:51:23.086822 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:23.088498 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 450/896] 10.222.0.18 () {56 vars in 1111 bytes} [Sat Aug 6 21:51:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:23.089403 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44a6a1e-fc70-4ab7-a56d-f89f0bbe0633 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:23.098686 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bde344d-43eb-4bbe-9256-7cb12941d010 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:23.121538 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:23.133976 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238cd50d-24da-4b91-b6a4-6ef37709853f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:23.134576 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238cd50d-24da-4b91-b6a4-6ef37709853f None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:23.134846 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238cd50d-24da-4b91-b6a4-6ef37709853f None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:23.135117 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238cd50d-24da-4b91-b6a4-6ef37709853f None admin] PATH_INFO: `/v3/services` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:23.135744 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238cd50d-24da-4b91-b6a4-6ef37709853f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:23.136049 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238cd50d-24da-4b91-b6a4-6ef37709853f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:23.137435 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238cd50d-24da-4b91-b6a4-6ef37709853f None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:23.142334 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 447/897] 10.222.0.18 () {56 vars in 1126 bytes} [Sat Aug 6 21:51:23 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:23.143106 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238cd50d-24da-4b91-b6a4-6ef37709853f None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:23.194864 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44a6a1e-fc70-4ab7-a56d-f89f0bbe0633 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:23.222235 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:23.227933 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2751da4-1075-418b-ad31-2fd0e7694872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:23.228572 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2751da4-1075-418b-ad31-2fd0e7694872 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:23.228860 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2751da4-1075-418b-ad31-2fd0e7694872 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:23.229118 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2751da4-1075-418b-ad31-2fd0e7694872 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:23.231120 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2751da4-1075-418b-ad31-2fd0e7694872 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:23.231647 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2751da4-1075-418b-ad31-2fd0e7694872 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:23.240506 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-c2751da4-1075-418b-ad31-2fd0e7694872 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 06 21:51:23.240506 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:23.240506 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 21:51:23.240506 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 21:51:23.240506 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 21:51:23.240506 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 21:51:23.240506 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 21:51:23.240506 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 21:51:23.240506 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 06 21:51:23.240506 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:23.240506 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 21:51:23.240506 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:23.240506 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:23.240506 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:23.240506 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:23.240506 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:23.240506 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:23.241515 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:23.241515 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:23.241515 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:23.241515 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:23.241515 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:23.241515 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:23.241515 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 21:51:23.241515 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 21:51:23.241515 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 21:51:23.241515 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 21:51:23.241515 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:23.241515 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:23.241515 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:23.241515 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:23.241515 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:23.241515 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:23.242383 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:23.242383 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:23.242383 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:23.242383 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:23.242383 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:23.242383 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:23.242383 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:23.242383 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:23.242383 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:23.242383 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:23.242383 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 21:51:23.242383 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 21:51:23.242383 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 06 21:51:23.242383 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:23.243137 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238cd50d-24da-4b91-b6a4-6ef37709853f None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:23.243212 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 451/898] 10.222.0.18 () {56 vars in 1126 bytes} [Sat Aug 6 21:51:23 2022] GET /identity/v3/services/object-store => generated 93 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:23.243212 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2751da4-1075-418b-ad31-2fd0e7694872 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:23.281085 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bbe155-96c7-4864-a6cf-10c0b1f2d3c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:23.281637 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bbe155-96c7-4864-a6cf-10c0b1f2d3c1 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:23.281938 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bbe155-96c7-4864-a6cf-10c0b1f2d3c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:23.282204 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bbe155-96c7-4864-a6cf-10c0b1f2d3c1 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:23.283180 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bbe155-96c7-4864-a6cf-10c0b1f2d3c1 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:23.284573 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bbe155-96c7-4864-a6cf-10c0b1f2d3c1 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:23.293892 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-00bbe155-96c7-4864-a6cf-10c0b1f2d3c1 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 06 21:51:23.293892 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:23.293892 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 21:51:23.293892 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 21:51:23.293892 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 21:51:23.293892 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 21:51:23.293892 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 21:51:23.293892 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 21:51:23.293892 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 06 21:51:23.293892 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:23.293892 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 21:51:23.293892 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:23.293892 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:23.293892 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:23.293892 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:23.293892 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:23.293892 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:23.295068 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:23.295068 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:23.295068 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:23.295068 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:23.295068 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:23.295068 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:23.295068 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 21:51:23.295068 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 21:51:23.295068 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 21:51:23.295068 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 21:51:23.295068 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:23.295068 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:23.295068 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:23.295068 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:23.295068 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:23.295068 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:23.296098 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:23.296098 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:23.296098 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:23.296098 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:23.296098 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:23.296098 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:23.296098 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:23.296098 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:23.296098 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:23.296098 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:23.296098 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 21:51:23.296098 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 21:51:23.296098 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 06 21:51:23.296098 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:23.296098 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 448/899] 10.222.0.18 () {56 vars in 1114 bytes} [Sat Aug 6 21:51:23 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:23.296933 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bbe155-96c7-4864-a6cf-10c0b1f2d3c1 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:23.338086 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:23.345032 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2751da4-1075-418b-ad31-2fd0e7694872 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:23.381694 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7927d688-855b-44de-bb37-8287420f1935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:23.382354 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7927d688-855b-44de-bb37-8287420f1935 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:23.382656 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7927d688-855b-44de-bb37-8287420f1935 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:23.382935 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7927d688-855b-44de-bb37-8287420f1935 None admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:23.383618 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7927d688-855b-44de-bb37-8287420f1935 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:23.383920 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7927d688-855b-44de-bb37-8287420f1935 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:23.385380 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7927d688-855b-44de-bb37-8287420f1935 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:23.397965 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 452/900] 10.222.0.18 () {56 vars in 1148 bytes} [Sat Aug 6 21:51:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:23.398771 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7927d688-855b-44de-bb37-8287420f1935 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:23.398872 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bbe155-96c7-4864-a6cf-10c0b1f2d3c1 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:23.433393 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8609634-7a22-494d-9a68-52b50a04cc6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:23.433993 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8609634-7a22-494d-9a68-52b50a04cc6b None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:23.434273 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8609634-7a22-494d-9a68-52b50a04cc6b None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:23.434539 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8609634-7a22-494d-9a68-52b50a04cc6b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:23.435235 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8609634-7a22-494d-9a68-52b50a04cc6b None admin] RBAC: Adding query filter params (interface=public, service_id=7055ee7d6f3546b598b8ab86e4170ee2, region_id=RegionOne) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:23.435540 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8609634-7a22-494d-9a68-52b50a04cc6b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:23.437528 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8609634-7a22-494d-9a68-52b50a04cc6b None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:23.442422 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 449/901] 10.222.0.18 () {56 vars in 1251 bytes} [Sat Aug 6 21:51:23 2022] GET /identity/v3/endpoints?service_id=7055ee7d6f3546b598b8ab86e4170ee2&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:23.443192 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8609634-7a22-494d-9a68-52b50a04cc6b None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:23.454498 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:23.499402 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7927d688-855b-44de-bb37-8287420f1935 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:23.532659 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5db105e-eb01-43ed-a68f-cef79c24ecc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:23.533181 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5db105e-eb01-43ed-a68f-cef79c24ecc1 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:23.533462 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5db105e-eb01-43ed-a68f-cef79c24ecc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:23.533730 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5db105e-eb01-43ed-a68f-cef79c24ecc1 None admin] PATH_INFO: `/v3/services` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:23.534351 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5db105e-eb01-43ed-a68f-cef79c24ecc1 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:23.534649 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5db105e-eb01-43ed-a68f-cef79c24ecc1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:23.536010 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5db105e-eb01-43ed-a68f-cef79c24ecc1 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:23.540449 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 453/902] 10.222.0.18 () {56 vars in 1122 bytes} [Sat Aug 6 21:51:23 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:23.541208 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5db105e-eb01-43ed-a68f-cef79c24ecc1 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:23.542991 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8609634-7a22-494d-9a68-52b50a04cc6b None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:23.563163 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:23.577386 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c39374-2e67-462b-84e1-0e4a3beee447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:23.589509 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c39374-2e67-462b-84e1-0e4a3beee447 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:23.589509 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c39374-2e67-462b-84e1-0e4a3beee447 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:23.589509 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c39374-2e67-462b-84e1-0e4a3beee447 None admin] PATH_INFO: `/v3/services` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:23.589509 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c39374-2e67-462b-84e1-0e4a3beee447 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:23.589509 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c39374-2e67-462b-84e1-0e4a3beee447 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:23.589509 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c39374-2e67-462b-84e1-0e4a3beee447 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:23.589990 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 450/903] 10.222.0.18 () {56 vars in 1114 bytes} [Sat Aug 6 21:51:23 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:23.589990 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c39374-2e67-462b-84e1-0e4a3beee447 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:23.643353 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5db105e-eb01-43ed-a68f-cef79c24ecc1 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:23.654083 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:23.678380 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1444d572-e9ad-4912-b7e2-d148be2e8766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:23.679074 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1444d572-e9ad-4912-b7e2-d148be2e8766 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:23.679358 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1444d572-e9ad-4912-b7e2-d148be2e8766 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:23.679630 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1444d572-e9ad-4912-b7e2-d148be2e8766 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:23.680343 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1444d572-e9ad-4912-b7e2-d148be2e8766 None admin] RBAC: Adding query filter params (role.id=08bf738879a941dab8ac6323043dc991, scope.project.id=4bb6307eb0644d85a1d1daaa61352697, user.id=d1b26ba592724dd389584bb800d877a2) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:23.680685 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1444d572-e9ad-4912-b7e2-d148be2e8766 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:23.682112 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1444d572-e9ad-4912-b7e2-d148be2e8766 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:23.685044 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c39374-2e67-462b-84e1-0e4a3beee447 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:23.697764 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 454/904] 10.222.0.18 () {56 vars in 1374 bytes} [Sat Aug 6 21:51:23 2022] GET /identity/v3/role_assignments?user.id=d1b26ba592724dd389584bb800d877a2&scope.project.id=4bb6307eb0644d85a1d1daaa61352697&role.id=08bf738879a941dab8ac6323043dc991 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:23.697764 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1444d572-e9ad-4912-b7e2-d148be2e8766 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:23.735176 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c974a22-1173-4ccc-a67d-6ce195cdc869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:23.735803 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c974a22-1173-4ccc-a67d-6ce195cdc869 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:23.736082 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c974a22-1173-4ccc-a67d-6ce195cdc869 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:23.736351 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c974a22-1173-4ccc-a67d-6ce195cdc869 None admin] PATH_INFO: `/v3/services` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:23.737009 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c974a22-1173-4ccc-a67d-6ce195cdc869 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:23.738428 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c974a22-1173-4ccc-a67d-6ce195cdc869 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:23.755123 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 451/905] 10.222.0.18 () {56 vars in 1087 bytes} [Sat Aug 6 21:51:23 2022] GET /identity/v3/services => generated 1268 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 21:51:23.755123 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c974a22-1173-4ccc-a67d-6ce195cdc869 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:23.801753 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:23.831292 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1444d572-e9ad-4912-b7e2-d148be2e8766 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:23.880008 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c974a22-1173-4ccc-a67d-6ce195cdc869 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:23.884448 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde62f8f-f485-403f-8bfc-f14fa0581d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:23.885423 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde62f8f-f485-403f-8bfc-f14fa0581d68 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:23.885884 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde62f8f-f485-403f-8bfc-f14fa0581d68 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:23.886330 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde62f8f-f485-403f-8bfc-f14fa0581d68 None admin] PATH_INFO: `/v3/services` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:23.887148 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde62f8f-f485-403f-8bfc-f14fa0581d68 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:23.887656 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde62f8f-f485-403f-8bfc-f14fa0581d68 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:23.889954 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde62f8f-f485-403f-8bfc-f14fa0581d68 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:23.897570 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 455/906] 10.222.0.18 () {56 vars in 1122 bytes} [Sat Aug 6 21:51:23 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:23.924168 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614d6315-a765-4baa-b88e-7a41ced625c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:23.924939 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614d6315-a765-4baa-b88e-7a41ced625c4 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:23.925222 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614d6315-a765-4baa-b88e-7a41ced625c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:23.925495 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614d6315-a765-4baa-b88e-7a41ced625c4 None admin] PATH_INFO: `/v3/services` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:23.926158 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614d6315-a765-4baa-b88e-7a41ced625c4 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:23.926460 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614d6315-a765-4baa-b88e-7a41ced625c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:23.927911 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614d6315-a765-4baa-b88e-7a41ced625c4 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:23.932803 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:23.940666 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 452/907] 10.222.0.18 () {56 vars in 1114 bytes} [Sat Aug 6 21:51:23 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:23.941695 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde62f8f-f485-403f-8bfc-f14fa0581d68 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:24.067064 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde62f8f-f485-403f-8bfc-f14fa0581d68 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:24.103975 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f3f4f4-b6f5-49b5-9b4c-b3914f093d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:24.104615 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f3f4f4-b6f5-49b5-9b4c-b3914f093d7b None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:24.104897 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f3f4f4-b6f5-49b5-9b4c-b3914f093d7b None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:24.105173 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f3f4f4-b6f5-49b5-9b4c-b3914f093d7b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:24.105873 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f3f4f4-b6f5-49b5-9b4c-b3914f093d7b None admin] RBAC: Adding query filter params (interface=public, service_id=1e53ee4f107247e6a3475788978818a2, region_id=RegionOne) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:24.106181 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f3f4f4-b6f5-49b5-9b4c-b3914f093d7b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:24.107827 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f3f4f4-b6f5-49b5-9b4c-b3914f093d7b None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:24.114044 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 456/908] 10.222.0.18 () {56 vars in 1251 bytes} [Sat Aug 6 21:51:23 2022] GET /identity/v3/endpoints?service_id=1e53ee4f107247e6a3475788978818a2&interface=public®ion_id=RegionOne => generated 194 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:24.119336 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614d6315-a765-4baa-b88e-7a41ced625c4 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:24.226177 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614d6315-a765-4baa-b88e-7a41ced625c4 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:24.262386 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59ea7e8-d578-4767-9885-83fdd0b8f972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:24.263367 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59ea7e8-d578-4767-9885-83fdd0b8f972 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:24.263869 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59ea7e8-d578-4767-9885-83fdd0b8f972 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:24.264931 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59ea7e8-d578-4767-9885-83fdd0b8f972 None admin] PATH_INFO: `/v3/services` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:24.265868 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59ea7e8-d578-4767-9885-83fdd0b8f972 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:24.267495 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59ea7e8-d578-4767-9885-83fdd0b8f972 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:24.274177 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 453/909] 10.222.0.18 () {56 vars in 1087 bytes} [Sat Aug 6 21:51:24 2022] GET /identity/v3/services => generated 1268 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 21:51:25.736916 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8e63c4-2c8f-4817-867c-c69173dbdb7e None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:25.737439 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8e63c4-2c8f-4817-867c-c69173dbdb7e None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:25.737439 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8e63c4-2c8f-4817-867c-c69173dbdb7e None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:25.739043 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 457/910] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:51:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:51:25.744230 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa097d6-a721-47ac-a9c6-0bf94935a5f0 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:25.744709 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa097d6-a721-47ac-a9c6-0bf94935a5f0 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:25.744787 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa097d6-a721-47ac-a9c6-0bf94935a5f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:25.763324 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-2aa097d6-a721-47ac-a9c6-0bf94935a5f0 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:51:25.765068 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-2aa097d6-a721-47ac-a9c6-0bf94935a5f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:51:25.826459 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:25.836347 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 454/911] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:51:25 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 21:51:25.859007 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8e63c4-2c8f-4817-867c-c69173dbdb7e None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:25.871650 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:25.892834 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac669e6-1903-4ff8-b5bd-934a47dea26e None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:25.893585 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac669e6-1903-4ff8-b5bd-934a47dea26e None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:25.894159 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac669e6-1903-4ff8-b5bd-934a47dea26e None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:25.897253 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 455/912] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:51:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:51:25.901617 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0189f118-304f-47a6-9f35-24ede01d93af None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:25.901899 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0189f118-304f-47a6-9f35-24ede01d93af None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:25.902173 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0189f118-304f-47a6-9f35-24ede01d93af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:25.923690 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-0189f118-304f-47a6-9f35-24ede01d93af None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:51:25.925344 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-0189f118-304f-47a6-9f35-24ede01d93af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:51:25.977765 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8e63c4-2c8f-4817-867c-c69173dbdb7e None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:26.013306 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 456/913] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:51:25 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 21:51:26.014607 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcde7b3-7c26-4879-902b-cee45e1f9142 None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:26.015027 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcde7b3-7c26-4879-902b-cee45e1f9142 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:26.015250 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcde7b3-7c26-4879-902b-cee45e1f9142 None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:26.016656 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d284055-612f-40f9-bb44-ceea5faffb77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:26.020651 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 457/914] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:51:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:51:26.020815 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d284055-612f-40f9-bb44-ceea5faffb77 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:26.020815 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d284055-612f-40f9-bb44-ceea5faffb77 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:26.020815 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d284055-612f-40f9-bb44-ceea5faffb77 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:26.022611 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b4d784-b9d4-4039-a240-1e495ba60704 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:26.022888 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b4d784-b9d4-4039-a240-1e495ba60704 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:26.023167 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b4d784-b9d4-4039-a240-1e495ba60704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:26.036941 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d284055-612f-40f9-bb44-ceea5faffb77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:51:26.042180 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-5d284055-612f-40f9-bb44-ceea5faffb77 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 06 21:51:26.042180 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:26.042180 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:26.042180 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:26.042180 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:26.042180 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:26.042180 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:26.042180 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:26.042180 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:26.042180 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:26.042180 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:26.042180 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:26.042180 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:51:26.042180 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:51:26.042180 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:51:26.042180 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:51:26.043166 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:51:26.043166 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:51:26.043166 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:51:26.043166 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:51:26.043166 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:26.043166 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:26.043166 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:26.043166 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:26.043166 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:26.043166 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:26.043166 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:26.043166 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:26.043166 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:26.043166 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:26.043166 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:26.043166 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:26.044068 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:26.044068 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:26.044068 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:26.044068 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:26.044068 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:51:26.044068 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:51:26.044068 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:51:26.044068 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:51:26.044068 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:51:26.044068 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:51:26.044068 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 06 21:51:26.044068 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:26.044068 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 458/915] 10.222.0.18 () {56 vars in 1120 bytes} [Sat Aug 6 21:51:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:26.045926 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d284055-612f-40f9-bb44-ceea5faffb77 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:26.048711 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-11b4d784-b9d4-4039-a240-1e495ba60704 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:51:26.048711 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-11b4d784-b9d4-4039-a240-1e495ba60704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:51:26.122263 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:26.125428 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 458/916] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:51:26 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 21:51:26.126275 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b4d784-b9d4-4039-a240-1e495ba60704 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:26.154253 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d284055-612f-40f9-bb44-ceea5faffb77 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:26.194253 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:26.197091 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e439d123-f130-4851-aef8-450afec8c033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:26.197715 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e439d123-f130-4851-aef8-450afec8c033 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:26.198023 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e439d123-f130-4851-aef8-450afec8c033 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:26.198253 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e439d123-f130-4851-aef8-450afec8c033 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:26.198885 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e439d123-f130-4851-aef8-450afec8c033 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:26.200178 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e439d123-f130-4851-aef8-450afec8c033 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:26.211835 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-e439d123-f130-4851-aef8-450afec8c033 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 06 21:51:26.211835 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:26.211835 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 21:51:26.211835 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 21:51:26.211835 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 21:51:26.211835 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 21:51:26.211835 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 21:51:26.211835 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 21:51:26.211835 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 06 21:51:26.211835 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:26.211835 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 21:51:26.211835 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:26.211835 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:26.211835 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:26.211835 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:26.211835 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:26.212873 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:26.212873 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:26.212873 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:26.212873 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:26.212873 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:26.212873 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:26.212873 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:26.212873 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 21:51:26.212873 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 21:51:26.212873 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 21:51:26.212873 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 21:51:26.212873 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:26.212873 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:26.212873 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:26.212873 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:26.212873 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:26.212873 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:26.214410 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:26.214410 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:26.214410 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:26.214410 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:26.214410 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:26.214410 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:26.214410 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:26.214410 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:26.214410 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:26.214410 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:26.214410 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 21:51:26.214410 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 21:51:26.214410 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 06 21:51:26.214410 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:26.214410 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 459/917] 10.222.0.18 () {56 vars in 1132 bytes} [Sat Aug 6 21:51:26 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:26.217853 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e439d123-f130-4851-aef8-450afec8c033 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:26.243911 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b4d784-b9d4-4039-a240-1e495ba60704 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:26.279007 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c9cce5-3592-4f5c-8737-103ed12426a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:26.279528 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c9cce5-3592-4f5c-8737-103ed12426a4 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:26.280456 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c9cce5-3592-4f5c-8737-103ed12426a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:26.280702 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c9cce5-3592-4f5c-8737-103ed12426a4 None admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:26.281364 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c9cce5-3592-4f5c-8737-103ed12426a4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:26.281800 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c9cce5-3592-4f5c-8737-103ed12426a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:26.283007 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c9cce5-3592-4f5c-8737-103ed12426a4 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:26.291715 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 459/918] 10.222.0.18 () {56 vars in 1115 bytes} [Sat Aug 6 21:51:26 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:26.292435 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c9cce5-3592-4f5c-8737-103ed12426a4 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:26.331100 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e439d123-f130-4851-aef8-450afec8c033 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:26.369217 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7775ec73-1dd6-47cd-9dfe-8f5c9172f316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:26.369824 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7775ec73-1dd6-47cd-9dfe-8f5c9172f316 None admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:26.370099 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7775ec73-1dd6-47cd-9dfe-8f5c9172f316 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:26.370373 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7775ec73-1dd6-47cd-9dfe-8f5c9172f316 None admin] PATH_INFO: `/v3/services` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:26.371009 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7775ec73-1dd6-47cd-9dfe-8f5c9172f316 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:26.372371 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7775ec73-1dd6-47cd-9dfe-8f5c9172f316 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:26.375748 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:26.398309 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c9cce5-3592-4f5c-8737-103ed12426a4 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:26.414380 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:26.431987 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996b4b2b-cc81-495e-bbf8-631ee6f9c1f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:26.432613 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b4b2b-cc81-495e-bbf8-631ee6f9c1f8 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:26.432889 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b4b2b-cc81-495e-bbf8-631ee6f9c1f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:26.433156 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b4b2b-cc81-495e-bbf8-631ee6f9c1f8 None admin] PATH_INFO: `/v3/services` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:26.433814 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996b4b2b-cc81-495e-bbf8-631ee6f9c1f8 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:26.434109 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996b4b2b-cc81-495e-bbf8-631ee6f9c1f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:26.435495 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996b4b2b-cc81-495e-bbf8-631ee6f9c1f8 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:26.439946 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 460/919] 10.222.0.18 () {56 vars in 1126 bytes} [Sat Aug 6 21:51:26 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:26.440829 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996b4b2b-cc81-495e-bbf8-631ee6f9c1f8 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:26.475257 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 460/920] 10.222.0.18 () {60 vars in 1141 bytes} [Sat Aug 6 21:51:26 2022] POST /identity/v3/services => generated 241 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 21:51:26.476584 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2c6bbd-5e76-4e2d-950b-08437d2cb602 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:26.476871 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2c6bbd-5e76-4e2d-950b-08437d2cb602 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:26.477144 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2c6bbd-5e76-4e2d-950b-08437d2cb602 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:26.478558 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 461/921] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:51:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:51:26.481333 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2c6bbd-5e76-4e2d-950b-08437d2cb602 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:26.554057 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996b4b2b-cc81-495e-bbf8-631ee6f9c1f8 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:26.598474 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effce88c-0d5a-4aa2-8aa6-7c60ad2c40dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:26.599112 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effce88c-0d5a-4aa2-8aa6-7c60ad2c40dc None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:26.599390 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effce88c-0d5a-4aa2-8aa6-7c60ad2c40dc None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:26.599656 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effce88c-0d5a-4aa2-8aa6-7c60ad2c40dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:26.604295 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2c6bbd-5e76-4e2d-950b-08437d2cb602 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:26.613307 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effce88c-0d5a-4aa2-8aa6-7c60ad2c40dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:26.614775 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effce88c-0d5a-4aa2-8aa6-7c60ad2c40dc None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:26.631237 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-effce88c-0d5a-4aa2-8aa6-7c60ad2c40dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:51:26.631237 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:26.631237 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:26.631237 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:26.631237 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:26.631237 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:26.631237 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:26.631237 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:26.631237 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:26.631237 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:26.631237 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:26.631237 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:26.631237 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:51:26.631237 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:51:26.631237 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:51:26.631237 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:51:26.632311 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:26.632311 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:26.632311 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:26.632311 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:26.632311 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:26.632311 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:26.632311 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:26.632311 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:26.632311 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:26.632311 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:26.632311 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:26.632311 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:26.632311 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:26.632311 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:26.632311 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:26.632311 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:26.632311 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:51:26.633309 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:51:26.633309 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:51:26.633309 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:26.633309 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 461/922] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:51:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:26.634054 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ba4a38-4884-4905-b143-090cd184ef45 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:26.634351 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ba4a38-4884-4905-b143-090cd184ef45 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:26.634590 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ba4a38-4884-4905-b143-090cd184ef45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:26.647196 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346252ef-1139-4a96-b811-2dc903900a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:26.648197 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346252ef-1139-4a96-b811-2dc903900a3a None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:26.648535 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346252ef-1139-4a96-b811-2dc903900a3a None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:26.649725 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346252ef-1139-4a96-b811-2dc903900a3a None admin] PATH_INFO: `/v3/services` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:26.649725 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346252ef-1139-4a96-b811-2dc903900a3a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:26.649725 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346252ef-1139-4a96-b811-2dc903900a3a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:26.651053 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346252ef-1139-4a96-b811-2dc903900a3a None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:26.652863 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-c3ba4a38-4884-4905-b143-090cd184ef45 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:51:26.654062 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-c3ba4a38-4884-4905-b143-090cd184ef45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:51:26.657841 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 462/923] 10.222.0.18 () {56 vars in 1126 bytes} [Sat Aug 6 21:51:26 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:26.658163 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346252ef-1139-4a96-b811-2dc903900a3a None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:26.700792 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:26.721659 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:26.729534 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 462/924] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:51:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 21:51:26.730359 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ba4a38-4884-4905-b143-090cd184ef45 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:26.775277 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346252ef-1139-4a96-b811-2dc903900a3a None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:26.816777 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fae1f48-d338-4e9c-8704-98c98d796d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:26.818133 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fae1f48-d338-4e9c-8704-98c98d796d63 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:26.818689 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fae1f48-d338-4e9c-8704-98c98d796d63 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:26.819560 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fae1f48-d338-4e9c-8704-98c98d796d63 None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:26.820693 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fae1f48-d338-4e9c-8704-98c98d796d63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:26.821283 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fae1f48-d338-4e9c-8704-98c98d796d63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:26.823778 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fae1f48-d338-4e9c-8704-98c98d796d63 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:26.839790 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 463/925] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:51:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:26.840304 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ba4a38-4884-4905-b143-090cd184ef45 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:26.841449 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fae1f48-d338-4e9c-8704-98c98d796d63 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:26.877484 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f81d3b-de5d-46d7-b5e3-9134c37a964b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:26.878199 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f81d3b-de5d-46d7-b5e3-9134c37a964b None admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:26.878544 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f81d3b-de5d-46d7-b5e3-9134c37a964b None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:26.878928 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f81d3b-de5d-46d7-b5e3-9134c37a964b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:26.879616 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f81d3b-de5d-46d7-b5e3-9134c37a964b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:26.881076 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f81d3b-de5d-46d7-b5e3-9134c37a964b None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:26.947256 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fae1f48-d338-4e9c-8704-98c98d796d63 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:26.964478 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:26.982464 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbfef25-d01e-4899-ae82-4c560e94bc0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:26.983650 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbfef25-d01e-4899-ae82-4c560e94bc0c None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:26.983925 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbfef25-d01e-4899-ae82-4c560e94bc0c None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:26.984195 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbfef25-d01e-4899-ae82-4c560e94bc0c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:26.984829 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbfef25-d01e-4899-ae82-4c560e94bc0c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:26.986206 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbfef25-d01e-4899-ae82-4c560e94bc0c None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:26.997206 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 463/926] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 21:51:26 2022] POST /identity/v3/endpoints => generated 347 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 21:51:26.997206 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f81d3b-de5d-46d7-b5e3-9134c37a964b None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:26.997502 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-adbfef25-d01e-4899-ae82-4c560e94bc0c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 06 21:51:26.997502 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:26.997502 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 21:51:26.997502 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 21:51:26.997502 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 21:51:26.997502 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 21:51:26.997502 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 21:51:26.997502 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 21:51:26.997502 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 06 21:51:26.997502 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:26.997502 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 21:51:26.997502 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:26.997502 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:26.997502 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:26.997502 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:26.997502 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:26.997502 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:26.998728 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:26.998728 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:26.998728 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:26.998728 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:26.998728 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:26.998728 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:26.998728 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 21:51:26.998728 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 21:51:26.998728 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 21:51:26.998728 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 21:51:26.998728 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:26.998728 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:26.998728 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:26.998728 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:26.998728 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:26.998728 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:27.000104 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:27.000104 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:27.000104 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:27.000104 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:27.000104 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:27.000104 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:27.000104 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:27.000104 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:27.000104 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:27.000104 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:27.000104 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 21:51:27.000104 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 21:51:27.000104 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 06 21:51:27.000104 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:27.004824 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 464/927] 10.222.0.18 () {56 vars in 1114 bytes} [Sat Aug 6 21:51:26 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:27.004824 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbfef25-d01e-4899-ae82-4c560e94bc0c None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:27.025877 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:27.133277 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbfef25-d01e-4899-ae82-4c560e94bc0c None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:27.180728 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f81d3b-de5d-46d7-b5e3-9134c37a964b None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:27.181077 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c73c3a7-1e32-4d5c-83d2-d9a9eab13320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:27.181689 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c73c3a7-1e32-4d5c-83d2-d9a9eab13320 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:27.181965 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c73c3a7-1e32-4d5c-83d2-d9a9eab13320 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:27.182235 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c73c3a7-1e32-4d5c-83d2-d9a9eab13320 None admin] PATH_INFO: `/v3/services` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:27.182864 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c73c3a7-1e32-4d5c-83d2-d9a9eab13320 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:27.183189 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c73c3a7-1e32-4d5c-83d2-d9a9eab13320 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:27.184596 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c73c3a7-1e32-4d5c-83d2-d9a9eab13320 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:27.191324 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 465/928] 10.222.0.18 () {56 vars in 1114 bytes} [Sat Aug 6 21:51:27 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:27.196942 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c73c3a7-1e32-4d5c-83d2-d9a9eab13320 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:27.222829 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd0e909-4b1b-4abf-ad73-00648e2c0e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:27.223459 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd0e909-4b1b-4abf-ad73-00648e2c0e5c None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:27.223734 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd0e909-4b1b-4abf-ad73-00648e2c0e5c None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:27.224003 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd0e909-4b1b-4abf-ad73-00648e2c0e5c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:27.234199 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd0e909-4b1b-4abf-ad73-00648e2c0e5c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:27.235654 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd0e909-4b1b-4abf-ad73-00648e2c0e5c None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:27.246152 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:27.249291 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-2fd0e909-4b1b-4abf-ad73-00648e2c0e5c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 06 21:51:27.249291 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:27.249291 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:27.249291 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:27.249291 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:27.249291 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:27.249291 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:27.249291 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:27.249291 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:27.249291 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:27.249291 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:27.249291 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:27.249291 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 21:51:27.249291 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 21:51:27.249291 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 21:51:27.249291 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 21:51:27.250388 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:27.250388 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:27.250388 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 21:51:27.250388 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 21:51:27.250388 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 21:51:27.250388 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 21:51:27.250388 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:27.250388 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:27.250388 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:27.250388 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:27.250388 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:27.250388 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:27.250388 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:27.250388 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:27.250388 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:27.250388 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:27.250388 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:27.250388 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:27.251865 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:27.251865 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:27.251865 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 21:51:27.251865 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 21:51:27.251865 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 21:51:27.251865 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 21:51:27.251865 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 21:51:27.251865 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 21:51:27.251865 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 06 21:51:27.251865 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:27.251865 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 464/929] 10.222.0.18 () {56 vars in 1117 bytes} [Sat Aug 6 21:51:26 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:27.258883 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd0e909-4b1b-4abf-ad73-00648e2c0e5c None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:27.314185 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c73c3a7-1e32-4d5c-83d2-d9a9eab13320 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:27.345111 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:27.349467 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc59318-0c49-42fa-9ee4-fa2077f02c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:27.350083 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc59318-0c49-42fa-9ee4-fa2077f02c67 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:27.350362 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc59318-0c49-42fa-9ee4-fa2077f02c67 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:27.350635 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc59318-0c49-42fa-9ee4-fa2077f02c67 None admin] PATH_INFO: `/v3/services` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:27.351264 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc59318-0c49-42fa-9ee4-fa2077f02c67 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:27.351563 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc59318-0c49-42fa-9ee4-fa2077f02c67 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:27.353059 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc59318-0c49-42fa-9ee4-fa2077f02c67 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:27.362183 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 466/930] 10.222.0.18 () {56 vars in 1114 bytes} [Sat Aug 6 21:51:27 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:51:27.365262 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc59318-0c49-42fa-9ee4-fa2077f02c67 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:27.384553 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd0e909-4b1b-4abf-ad73-00648e2c0e5c None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:27.424215 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c239ffab-3469-4cce-af87-59b6a84d09c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:27.425281 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c239ffab-3469-4cce-af87-59b6a84d09c4 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:27.426042 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c239ffab-3469-4cce-af87-59b6a84d09c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:27.426600 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c239ffab-3469-4cce-af87-59b6a84d09c4 None admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:27.427844 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c239ffab-3469-4cce-af87-59b6a84d09c4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:27.428542 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c239ffab-3469-4cce-af87-59b6a84d09c4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:27.430496 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c239ffab-3469-4cce-af87-59b6a84d09c4 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:27.450984 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 465/931] 10.222.0.18 () {56 vars in 1149 bytes} [Sat Aug 6 21:51:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:51:27.457148 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c239ffab-3469-4cce-af87-59b6a84d09c4 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:27.473194 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc59318-0c49-42fa-9ee4-fa2077f02c67 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:27.480616 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:27.518321 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0b8380-fab4-4392-ad28-4388aa21538d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:27.520045 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0b8380-fab4-4392-ad28-4388aa21538d None admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:27.520580 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0b8380-fab4-4392-ad28-4388aa21538d None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:27.521103 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0b8380-fab4-4392-ad28-4388aa21538d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:27.522589 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0b8380-fab4-4392-ad28-4388aa21538d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:27.524495 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0b8380-fab4-4392-ad28-4388aa21538d None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:27.579159 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c239ffab-3469-4cce-af87-59b6a84d09c4 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:27.608958 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:27.617007 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82008a0a-14d5-4530-ab8c-f720150008a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:27.617656 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82008a0a-14d5-4530-ab8c-f720150008a9 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:27.617908 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82008a0a-14d5-4530-ab8c-f720150008a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:27.618270 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82008a0a-14d5-4530-ab8c-f720150008a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:27.618713 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 467/932] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 21:51:27 2022] POST /identity/v3/endpoints => generated 346 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 21:51:27.638865 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82008a0a-14d5-4530-ab8c-f720150008a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:27.641254 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82008a0a-14d5-4530-ab8c-f720150008a9 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:27.661224 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-82008a0a-14d5-4530-ab8c-f720150008a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:51:27.661224 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:27.661224 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:27.661224 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:27.661224 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:27.661224 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:27.661224 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:27.661224 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:27.661224 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:27.661224 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:27.661224 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:27.661224 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:27.661224 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:51:27.661224 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:51:27.661224 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:51:27.661224 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:51:27.662611 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:27.662611 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:27.662611 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:27.662611 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:27.662611 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:27.662611 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:27.662611 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:27.662611 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:27.662611 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:27.662611 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:27.662611 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:27.662611 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:27.662611 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:27.662611 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:27.662611 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:27.662611 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:27.662611 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:51:27.663612 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:51:27.663612 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:51:27.663612 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:27.666969 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 466/933] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:51:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 21:51:27.675488 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0b8380-fab4-4392-ad28-4388aa21538d None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:27.801237 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0b8380-fab4-4392-ad28-4388aa21538d None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:27.840961 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5cbcdf-4544-485a-bfa7-7fe9169ab23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:27.841665 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5cbcdf-4544-485a-bfa7-7fe9169ab23a None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:27.841948 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5cbcdf-4544-485a-bfa7-7fe9169ab23a None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:27.842765 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5cbcdf-4544-485a-bfa7-7fe9169ab23a None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:27.843619 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5cbcdf-4544-485a-bfa7-7fe9169ab23a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:27.843953 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5cbcdf-4544-485a-bfa7-7fe9169ab23a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:27.845330 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5cbcdf-4544-485a-bfa7-7fe9169ab23a None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:27.858580 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 468/934] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:51:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:27.864069 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82008a0a-14d5-4530-ab8c-f720150008a9 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:27.976529 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82008a0a-14d5-4530-ab8c-f720150008a9 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:28.013620 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4f8df3-f18b-4515-a4e8-1f5ac50c373b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:28.014373 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4f8df3-f18b-4515-a4e8-1f5ac50c373b None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:28.014715 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4f8df3-f18b-4515-a4e8-1f5ac50c373b None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:28.015051 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4f8df3-f18b-4515-a4e8-1f5ac50c373b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:28.022391 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4f8df3-f18b-4515-a4e8-1f5ac50c373b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:28.023923 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4f8df3-f18b-4515-a4e8-1f5ac50c373b None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:28.035850 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-5b4f8df3-f18b-4515-a4e8-1f5ac50c373b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 21:51:28.035850 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:28.035850 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:28.035850 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:28.035850 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:28.035850 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:28.035850 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:28.035850 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:28.035850 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:28.035850 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:28.035850 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:28.035850 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:28.035850 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 21:51:28.035850 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 21:51:28.035850 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 21:51:28.036985 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 21:51:28.036985 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:28.036985 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:28.036985 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:28.036985 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:28.036985 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:28.036985 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:28.036985 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:28.036985 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:28.036985 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:28.036985 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:28.036985 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:28.036985 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:28.036985 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:28.036985 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:28.036985 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:28.036985 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:28.038099 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 21:51:28.038099 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 21:51:28.038099 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 21:51:28.038099 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 21:51:28.038099 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 21:51:28.038099 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 21:51:28.038099 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 21:51:28.038099 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:28.039782 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 467/935] 10.222.0.18 () {56 vars in 1111 bytes} [Sat Aug 6 21:51:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:28.046125 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5cbcdf-4544-485a-bfa7-7fe9169ab23a None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:28.067959 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:28.152284 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5cbcdf-4544-485a-bfa7-7fe9169ab23a None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:28.190371 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0fafb3-0e4c-4924-8e3a-8e70b9065e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:28.191238 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0fafb3-0e4c-4924-8e3a-8e70b9065e49 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:28.191656 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0fafb3-0e4c-4924-8e3a-8e70b9065e49 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:28.192070 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0fafb3-0e4c-4924-8e3a-8e70b9065e49 None admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:28.192931 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0fafb3-0e4c-4924-8e3a-8e70b9065e49 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:28.193391 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0fafb3-0e4c-4924-8e3a-8e70b9065e49 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:28.194944 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0fafb3-0e4c-4924-8e3a-8e70b9065e49 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:28.208511 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 469/936] 10.222.0.18 () {56 vars in 1148 bytes} [Sat Aug 6 21:51:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:28.214530 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4f8df3-f18b-4515-a4e8-1f5ac50c373b None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:28.261754 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:28.327250 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4f8df3-f18b-4515-a4e8-1f5ac50c373b None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:28.367360 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27caa4d2-6a31-4388-98ae-157baa48ebb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:28.368739 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27caa4d2-6a31-4388-98ae-157baa48ebb9 None admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:28.369224 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27caa4d2-6a31-4388-98ae-157baa48ebb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:28.369716 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27caa4d2-6a31-4388-98ae-157baa48ebb9 None admin] PATH_INFO: `/v3/projects/4bb6307eb0644d85a1d1daaa61352697/users/d1b26ba592724dd389584bb800d877a2/roles/08bf738879a941dab8ac6323043dc991` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:28.372138 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27caa4d2-6a31-4388-98ae-157baa48ebb9 None admin] RBAC: Authorizing `identity:create_grant(project_id=4bb6307eb0644d85a1d1daaa61352697, user_id=d1b26ba592724dd389584bb800d877a2, role_id=08bf738879a941dab8ac6323043dc991)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:28.374332 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27caa4d2-6a31-4388-98ae-157baa48ebb9 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:28.447827 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 468/937] 10.222.0.18 () {58 vars in 1439 bytes} [Sat Aug 6 21:51:28 2022] PUT /identity/v3/projects/4bb6307eb0644d85a1d1daaa61352697/users/d1b26ba592724dd389584bb800d877a2/roles/08bf738879a941dab8ac6323043dc991 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 21:51:28.584797 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c99447-f6db-47a7-9bcd-da727c9f154c None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:28.584994 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c99447-f6db-47a7-9bcd-da727c9f154c None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:28.585236 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c99447-f6db-47a7-9bcd-da727c9f154c None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:28.586848 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 470/938] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:51:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:51:28.594368 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c68ffe-906b-40ce-b0c0-0266595347fc None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:28.594851 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c68ffe-906b-40ce-b0c0-0266595347fc None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:28.595292 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c68ffe-906b-40ce-b0c0-0266595347fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:28.616663 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-a4c68ffe-906b-40ce-b0c0-0266595347fc None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:51:28.618081 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-a4c68ffe-906b-40ce-b0c0-0266595347fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:51:28.692874 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 469/939] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:51:28 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 21:51:28.715136 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c99447-f6db-47a7-9bcd-da727c9f154c None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:28.802570 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:28.825839 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c99447-f6db-47a7-9bcd-da727c9f154c None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:28.865608 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4574263b-bbd7-4467-990e-72844e2d2f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:28.866161 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4574263b-bbd7-4467-990e-72844e2d2f9c None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:28.866502 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4574263b-bbd7-4467-990e-72844e2d2f9c None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:28.866780 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4574263b-bbd7-4467-990e-72844e2d2f9c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:28.867372 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4574263b-bbd7-4467-990e-72844e2d2f9c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:28.868780 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4574263b-bbd7-4467-990e-72844e2d2f9c None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:28.878080 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-4574263b-bbd7-4467-990e-72844e2d2f9c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 06 21:51:28.878080 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:28.878080 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 21:51:28.878080 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 21:51:28.878080 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 21:51:28.878080 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 21:51:28.878080 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 21:51:28.878080 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 21:51:28.878080 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 06 21:51:28.878080 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:28.878080 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 21:51:28.878080 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:28.878080 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:28.878080 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:28.878080 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:28.878080 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:28.878080 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:28.879085 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:28.879085 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:28.879085 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:28.879085 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:28.879085 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:28.879085 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:28.879085 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 21:51:28.879085 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 21:51:28.879085 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 21:51:28.879085 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 21:51:28.879085 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:28.879085 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:28.879085 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:28.879085 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:28.879085 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:28.879085 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:28.879916 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:28.879916 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:28.879916 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:28.879916 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:28.879916 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:28.879916 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:28.879916 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:28.879916 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:28.879916 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:28.879916 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:28.879916 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 21:51:28.879916 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 21:51:28.879916 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 06 21:51:28.879916 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:28.879916 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 471/940] 10.222.0.18 () {56 vars in 1126 bytes} [Sat Aug 6 21:51:28 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:28.887188 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c68ffe-906b-40ce-b0c0-0266595347fc None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:28.919367 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:28.994401 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c68ffe-906b-40ce-b0c0-0266595347fc None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:29.030430 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c7e183-8942-46fc-ae16-8aec4d6efa56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:29.031036 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c7e183-8942-46fc-ae16-8aec4d6efa56 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:29.031315 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c7e183-8942-46fc-ae16-8aec4d6efa56 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:29.031591 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c7e183-8942-46fc-ae16-8aec4d6efa56 None admin] PATH_INFO: `/v3/services` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:29.032217 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c7e183-8942-46fc-ae16-8aec4d6efa56 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:29.032549 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c7e183-8942-46fc-ae16-8aec4d6efa56 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:29.033913 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c7e183-8942-46fc-ae16-8aec4d6efa56 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:29.039011 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 470/941] 10.222.0.18 () {56 vars in 1122 bytes} [Sat Aug 6 21:51:28 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:29.044557 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4574263b-bbd7-4467-990e-72844e2d2f9c None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:29.150220 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4574263b-bbd7-4467-990e-72844e2d2f9c None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:29.186537 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad23572-1ee8-46bc-bb45-504a901115a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:29.187511 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad23572-1ee8-46bc-bb45-504a901115a0 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:29.187995 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad23572-1ee8-46bc-bb45-504a901115a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:29.188504 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad23572-1ee8-46bc-bb45-504a901115a0 None admin] PATH_INFO: `/v3/services` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:29.189338 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad23572-1ee8-46bc-bb45-504a901115a0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:29.189849 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad23572-1ee8-46bc-bb45-504a901115a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:29.191410 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad23572-1ee8-46bc-bb45-504a901115a0 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:29.196986 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 472/942] 10.222.0.18 () {56 vars in 1122 bytes} [Sat Aug 6 21:51:29 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:29.203191 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c7e183-8942-46fc-ae16-8aec4d6efa56 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:29.230946 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b1e490-7997-490d-bb30-486f58891a1d None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:29.231555 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b1e490-7997-490d-bb30-486f58891a1d None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:29.232288 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b1e490-7997-490d-bb30-486f58891a1d None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:29.234280 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 473/943] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:51:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:51:29.239327 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa55b36a-9eab-4521-a0f6-538bf4f315a8 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:29.239756 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa55b36a-9eab-4521-a0f6-538bf4f315a8 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:29.240279 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa55b36a-9eab-4521-a0f6-538bf4f315a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:29.253982 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:29.261476 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-aa55b36a-9eab-4521-a0f6-538bf4f315a8 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:51:29.263376 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-aa55b36a-9eab-4521-a0f6-538bf4f315a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:51:29.315913 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:29.323355 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c7e183-8942-46fc-ae16-8aec4d6efa56 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:29.359524 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 474/944] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:51:29 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 21:51:29.368517 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466a2418-d72b-4a13-a62a-5154615e4762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:29.369894 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466a2418-d72b-4a13-a62a-5154615e4762 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:29.370506 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466a2418-d72b-4a13-a62a-5154615e4762 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:29.371100 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466a2418-d72b-4a13-a62a-5154615e4762 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:29.372445 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466a2418-d72b-4a13-a62a-5154615e4762 None admin] RBAC: Adding query filter params (interface=public, service_id=557ace9e3e3543aaada4d3cfe53af3b2, region_id=RegionOne) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:29.372972 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466a2418-d72b-4a13-a62a-5154615e4762 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:29.375024 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466a2418-d72b-4a13-a62a-5154615e4762 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:29.380898 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 471/945] 10.222.0.18 () {56 vars in 1251 bytes} [Sat Aug 6 21:51:29 2022] GET /identity/v3/endpoints?service_id=557ace9e3e3543aaada4d3cfe53af3b2&interface=public®ion_id=RegionOne => generated 194 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:51:29.386429 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466a2418-d72b-4a13-a62a-5154615e4762 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:29.386663 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa55b36a-9eab-4521-a0f6-538bf4f315a8 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:29.521384 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466a2418-d72b-4a13-a62a-5154615e4762 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:29.525377 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa55b36a-9eab-4521-a0f6-538bf4f315a8 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:29.564581 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45cf0e7-0d8c-4955-a905-1b3dc9b9c7dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:29.565353 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45cf0e7-0d8c-4955-a905-1b3dc9b9c7dd None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:29.565732 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45cf0e7-0d8c-4955-a905-1b3dc9b9c7dd None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:29.566094 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45cf0e7-0d8c-4955-a905-1b3dc9b9c7dd None admin] PATH_INFO: `/v3/services` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:29.567006 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45cf0e7-0d8c-4955-a905-1b3dc9b9c7dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:29.569241 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45cf0e7-0d8c-4955-a905-1b3dc9b9c7dd None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:29.573358 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2183de0f-998d-428e-8ab9-a31d00073568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:29.574174 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2183de0f-998d-428e-8ab9-a31d00073568 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:29.574600 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2183de0f-998d-428e-8ab9-a31d00073568 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:29.575024 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2183de0f-998d-428e-8ab9-a31d00073568 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:29.575755 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2183de0f-998d-428e-8ab9-a31d00073568 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:29.582384 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 472/946] 10.222.0.18 () {56 vars in 1087 bytes} [Sat Aug 6 21:51:29 2022] GET /identity/v3/services => generated 1497 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 21:51:29.582605 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2183de0f-998d-428e-8ab9-a31d00073568 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:29.594190 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-2183de0f-998d-428e-8ab9-a31d00073568 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 06 21:51:29.594190 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:29.594190 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 21:51:29.594190 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 21:51:29.594190 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 21:51:29.594190 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 21:51:29.594190 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 21:51:29.594190 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 21:51:29.594190 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 06 21:51:29.594190 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:29.594190 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 21:51:29.594190 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:29.594190 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:29.594190 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:29.594190 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:29.594190 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:29.594190 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:29.595281 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:29.595281 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:29.595281 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:29.595281 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:29.595281 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:29.595281 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:29.595281 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 21:51:29.595281 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 21:51:29.595281 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 21:51:29.595281 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 21:51:29.595281 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:29.595281 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:29.595281 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:29.595281 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:29.595281 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:29.595281 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:29.596163 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:29.596163 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:29.596163 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:29.596163 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:29.596163 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:29.596163 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:29.596163 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:29.596163 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:29.596163 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:29.596163 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:29.596163 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 21:51:29.596163 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 21:51:29.596163 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 06 21:51:29.596163 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:29.596163 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 475/947] 10.222.0.18 () {56 vars in 1111 bytes} [Sat Aug 6 21:51:29 2022] GET /identity/v3/services/compute => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:29.597794 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:29.602388 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45cf0e7-0d8c-4955-a905-1b3dc9b9c7dd None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:29.632391 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:29.718096 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45cf0e7-0d8c-4955-a905-1b3dc9b9c7dd None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:29.757502 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c9c548-4c1c-4be1-ac01-e8bdb3a6af6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:29.758122 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c9c548-4c1c-4be1-ac01-e8bdb3a6af6c None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:29.758403 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c9c548-4c1c-4be1-ac01-e8bdb3a6af6c None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:29.758680 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c9c548-4c1c-4be1-ac01-e8bdb3a6af6c None admin] PATH_INFO: `/v3/services` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:29.759329 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c9c548-4c1c-4be1-ac01-e8bdb3a6af6c None admin] RBAC: Adding query filter params (name=compute) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:29.759632 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c9c548-4c1c-4be1-ac01-e8bdb3a6af6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:29.761054 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c9c548-4c1c-4be1-ac01-e8bdb3a6af6c None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:29.766769 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 473/948] 10.222.0.18 () {56 vars in 1112 bytes} [Sat Aug 6 21:51:29 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:29.772692 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2183de0f-998d-428e-8ab9-a31d00073568 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:29.815670 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d74753-f73b-44e3-8ef7-3ab536ddb6b5 None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:29.815670 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d74753-f73b-44e3-8ef7-3ab536ddb6b5 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:29.815670 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d74753-f73b-44e3-8ef7-3ab536ddb6b5 None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:29.816217 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 474/949] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:51:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 21:51:29.821779 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40372b8-3d0c-4468-b37e-d3549316bddc None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:29.822086 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40372b8-3d0c-4468-b37e-d3549316bddc None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:29.822364 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40372b8-3d0c-4468-b37e-d3549316bddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:29.842958 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-c40372b8-3d0c-4468-b37e-d3549316bddc None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:51:29.844232 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-c40372b8-3d0c-4468-b37e-d3549316bddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:51:29.888061 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2183de0f-998d-428e-8ab9-a31d00073568 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:29.924448 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 475/950] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:51:29 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 21:51:29.938119 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fd2447-6708-4ab0-941e-7ddaa09bab91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:29.939074 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd2447-6708-4ab0-941e-7ddaa09bab91 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:29.939561 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd2447-6708-4ab0-941e-7ddaa09bab91 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:29.940031 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd2447-6708-4ab0-941e-7ddaa09bab91 None admin] PATH_INFO: `/v3/services` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:29.941020 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fd2447-6708-4ab0-941e-7ddaa09bab91 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:29.941523 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fd2447-6708-4ab0-941e-7ddaa09bab91 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:29.943191 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fd2447-6708-4ab0-941e-7ddaa09bab91 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:29.945859 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40372b8-3d0c-4468-b37e-d3549316bddc None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:29.953586 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 476/951] 10.222.0.18 () {56 vars in 1112 bytes} [Sat Aug 6 21:51:29 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:29.968451 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:30.065225 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40372b8-3d0c-4468-b37e-d3549316bddc None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:30.104763 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41985c36-dbdd-4d3c-8f52-24e592da7cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:30.105732 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41985c36-dbdd-4d3c-8f52-24e592da7cb0 None admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:30.106284 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41985c36-dbdd-4d3c-8f52-24e592da7cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:30.106777 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41985c36-dbdd-4d3c-8f52-24e592da7cb0 None admin] PATH_INFO: `/v3/projects` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:30.108539 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41985c36-dbdd-4d3c-8f52-24e592da7cb0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:30.110293 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41985c36-dbdd-4d3c-8f52-24e592da7cb0 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:30.113131 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: WARNING keystone.server.flask.common [None req-41985c36-dbdd-4d3c-8f52-24e592da7cb0 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 06 21:51:30.172301 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.sql.core [None req-41985c36-dbdd-4d3c-8f52-24e592da7cb0 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 06 21:51:30.172301 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 06 21:51:30.172301 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [parameters: {'id': '6cbaa7e75e9246f2b9c4ff0253acba87', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 06 21:51:30.172301 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80423) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 06 21:51:30.180363 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-41985c36-dbdd-4d3c-8f52-24e592da7cb0 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 6cbaa7e75e9246f2b9c4ff0253acba87.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 6cbaa7e75e9246f2b9c4ff0253acba87. Aug 06 21:51:30.180363 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:30.180363 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 06 21:51:30.180363 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 06 21:51:30.180363 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 06 21:51:30.180363 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 06 21:51:30.180363 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 06 21:51:30.180363 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application result = self._query(query) Aug 06 21:51:30.180363 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 06 21:51:30.180363 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application conn.query(q) Aug 06 21:51:30.180363 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 06 21:51:30.180363 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 06 21:51:30.180363 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 06 21:51:30.181174 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application result.read() Aug 06 21:51:30.181174 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 06 21:51:30.181174 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 06 21:51:30.181174 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 06 21:51:30.181174 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 06 21:51:30.181174 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 06 21:51:30.181174 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 06 21:51:30.181174 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 06 21:51:30.181174 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 06 21:51:30.181174 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 06 21:51:30.181174 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:30.181174 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 06 21:51:30.181174 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:30.181174 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:30.181174 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 06 21:51:30.181174 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 06 21:51:30.181174 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 06 21:51:30.181174 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 06 21:51:30.182123 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 06 21:51:30.182123 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application next(self.gen) Aug 06 21:51:30.182123 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 06 21:51:30.182123 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application yield resource Aug 06 21:51:30.182123 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 06 21:51:30.182123 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application next(self.gen) Aug 06 21:51:30.182123 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 06 21:51:30.182123 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application self.session.rollback() Aug 06 21:51:30.182123 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 06 21:51:30.182123 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application self.force_reraise() Aug 06 21:51:30.182123 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 06 21:51:30.182123 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise self.value Aug 06 21:51:30.182123 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 06 21:51:30.182123 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 06 21:51:30.182123 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 06 21:51:30.182123 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application session.commit() Aug 06 21:51:30.182123 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 06 21:51:30.183007 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 06 21:51:30.183007 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 06 21:51:30.183007 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application self._prepare_impl() Aug 06 21:51:30.183007 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 06 21:51:30.183007 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application self.session.flush() Aug 06 21:51:30.183007 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 06 21:51:30.183007 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application self._flush(objects) Aug 06 21:51:30.183007 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 06 21:51:30.183007 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 06 21:51:30.183007 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 06 21:51:30.183007 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application compat.raise_( Aug 06 21:51:30.183007 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 06 21:51:30.183007 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception Aug 06 21:51:30.183007 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 06 21:51:30.183007 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application flush_context.execute() Aug 06 21:51:30.183007 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 06 21:51:30.183007 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rec.execute(self) Aug 06 21:51:30.183908 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 06 21:51:30.183908 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 06 21:51:30.183908 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 06 21:51:30.183908 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 06 21:51:30.183908 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 06 21:51:30.183908 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 06 21:51:30.183908 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 06 21:51:30.183908 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 06 21:51:30.183908 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 06 21:51:30.183908 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 06 21:51:30.183908 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 06 21:51:30.183908 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 06 21:51:30.183908 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 06 21:51:30.183908 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 06 21:51:30.183908 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 06 21:51:30.184745 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 06 21:51:30.184745 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 06 21:51:30.184745 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception Aug 06 21:51:30.184745 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 06 21:51:30.184745 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 06 21:51:30.184745 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 06 21:51:30.184745 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 06 21:51:30.184745 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 06 21:51:30.184745 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application result = self._query(query) Aug 06 21:51:30.184745 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 06 21:51:30.184745 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application conn.query(q) Aug 06 21:51:30.184745 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 06 21:51:30.184745 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 06 21:51:30.184745 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 06 21:51:30.184745 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application result.read() Aug 06 21:51:30.184745 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 06 21:51:30.184745 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 06 21:51:30.185607 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 06 21:51:30.185607 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 06 21:51:30.185607 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 06 21:51:30.185607 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 06 21:51:30.185607 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 06 21:51:30.185607 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 06 21:51:30.185607 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 06 21:51:30.185607 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 06 21:51:30.185607 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [parameters: {'id': '6cbaa7e75e9246f2b9c4ff0253acba87', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 06 21:51:30.185607 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 06 21:51:30.185607 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:30.185607 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 21:51:30.185607 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:30.185607 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:30.185607 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 06 21:51:30.186414 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 06 21:51:30.186414 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 06 21:51:30.186414 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 06 21:51:30.186414 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 06 21:51:30.186414 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:30.186414 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 21:51:30.186414 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:30.186414 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:30.186414 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:30.186414 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:30.186414 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:30.186414 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:30.186414 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:30.186414 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:30.186414 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:30.186414 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:30.186414 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:30.187326 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:30.187326 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 06 21:51:30.187326 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 06 21:51:30.187326 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:30.187326 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:30.187326 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 06 21:51:30.187326 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 06 21:51:30.187326 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 6cbaa7e75e9246f2b9c4ff0253acba87. Aug 06 21:51:30.187326 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:30.190950 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 476/952] 10.222.0.18 () {60 vars in 1140 bytes} [Sat Aug 6 21:51:29 2022] POST /identity/v3/projects => generated 262 bytes in 245 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:30.198588 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fd2447-6708-4ab0-941e-7ddaa09bab91 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:30.262812 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:30.302389 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fd2447-6708-4ab0-941e-7ddaa09bab91 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:30.337113 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf006a49-3d6e-4dda-b5e7-c8dbe80968cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:30.337661 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf006a49-3d6e-4dda-b5e7-c8dbe80968cf None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:30.337938 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf006a49-3d6e-4dda-b5e7-c8dbe80968cf None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:30.338208 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf006a49-3d6e-4dda-b5e7-c8dbe80968cf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:30.345610 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf006a49-3d6e-4dda-b5e7-c8dbe80968cf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:30.347065 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf006a49-3d6e-4dda-b5e7-c8dbe80968cf None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:30.359434 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-cf006a49-3d6e-4dda-b5e7-c8dbe80968cf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 21:51:30.359434 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:30.359434 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:30.359434 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:30.359434 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:30.359434 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:30.359434 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:30.359434 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:30.359434 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:30.359434 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:30.359434 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:30.359434 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:30.359434 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 21:51:30.359434 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 21:51:30.359434 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 21:51:30.360569 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 21:51:30.360569 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:30.360569 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:30.360569 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:30.360569 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:30.360569 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:30.360569 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:30.360569 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:30.360569 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:30.360569 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:30.360569 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:30.360569 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:30.360569 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:30.360569 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:30.360569 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:30.360569 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:30.360569 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:30.361532 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 21:51:30.361532 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 21:51:30.361532 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 21:51:30.361532 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 21:51:30.361532 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 21:51:30.361532 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 21:51:30.361532 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 21:51:30.361532 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:30.361532 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 477/953] 10.222.0.18 () {56 vars in 1111 bytes} [Sat Aug 6 21:51:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:30.366826 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41985c36-dbdd-4d3c-8f52-24e592da7cb0 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:30.446227 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:30.478218 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41985c36-dbdd-4d3c-8f52-24e592da7cb0 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:30.515643 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a616b926-6b80-4515-9cff-7ef73b323f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:30.516249 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a616b926-6b80-4515-9cff-7ef73b323f9b None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:30.516573 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a616b926-6b80-4515-9cff-7ef73b323f9b None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:30.516845 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a616b926-6b80-4515-9cff-7ef73b323f9b None admin] PATH_INFO: `/v3/projects` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:30.517509 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a616b926-6b80-4515-9cff-7ef73b323f9b None admin] RBAC: Adding query filter params (name=service) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:30.517805 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a616b926-6b80-4515-9cff-7ef73b323f9b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:30.519556 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a616b926-6b80-4515-9cff-7ef73b323f9b None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:30.538475 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051e1a74-412c-46cc-91d4-1e731241d0c2 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:30.538793 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051e1a74-412c-46cc-91d4-1e731241d0c2 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:30.539049 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051e1a74-412c-46cc-91d4-1e731241d0c2 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:30.539994 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 477/954] 10.222.0.18 () {56 vars in 1112 bytes} [Sat Aug 6 21:51:30 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:30.544536 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 478/955] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:51:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:51:30.547631 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134fe6b2-115d-4206-b293-13987152c289 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:30.547917 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134fe6b2-115d-4206-b293-13987152c289 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:30.548181 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134fe6b2-115d-4206-b293-13987152c289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:30.570790 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-134fe6b2-115d-4206-b293-13987152c289 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:51:30.572238 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-134fe6b2-115d-4206-b293-13987152c289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:51:30.646710 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 478/956] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:51:30 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 21:51:30.670686 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051e1a74-412c-46cc-91d4-1e731241d0c2 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:30.788008 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051e1a74-412c-46cc-91d4-1e731241d0c2 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:30.812961 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:30.823668 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af8b55d-c390-4546-b0eb-d07dbc1f4591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:30.840789 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af8b55d-c390-4546-b0eb-d07dbc1f4591 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:30.840789 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af8b55d-c390-4546-b0eb-d07dbc1f4591 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:30.840789 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af8b55d-c390-4546-b0eb-d07dbc1f4591 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:30.840789 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af8b55d-c390-4546-b0eb-d07dbc1f4591 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:51:30.840789 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-6af8b55d-c390-4546-b0eb-d07dbc1f4591 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 06 21:51:30.840789 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:30.840789 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:30.840789 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:30.840789 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:30.841387 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:30.841387 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:30.841387 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:30.841387 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:30.841387 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:30.841387 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:30.841387 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:30.841387 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:51:30.841387 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:51:30.841387 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:51:30.841387 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:51:30.841387 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:51:30.841387 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:51:30.841387 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:51:30.841387 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:51:30.841387 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:30.843298 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:30.843298 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:30.843298 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:30.843298 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:30.843298 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:30.843298 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:30.843298 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:30.843298 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:30.843298 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:30.843298 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:30.843298 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:30.843298 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:30.843298 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:30.843298 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:30.843298 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:30.843298 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:51:30.843298 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:51:30.844196 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:51:30.844196 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:51:30.844196 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:51:30.844196 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:51:30.844196 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 06 21:51:30.844196 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:30.844196 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 479/957] 10.222.0.18 () {56 vars in 1120 bytes} [Sat Aug 6 21:51:30 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:30.852084 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134fe6b2-115d-4206-b293-13987152c289 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:30.922540 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:30.962319 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134fe6b2-115d-4206-b293-13987152c289 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:30.997703 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fb693d-bed4-4e95-89d6-281c802e36b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:30.999023 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fb693d-bed4-4e95-89d6-281c802e36b9 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:30.999529 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fb693d-bed4-4e95-89d6-281c802e36b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:31.000000 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fb693d-bed4-4e95-89d6-281c802e36b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:31.000902 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fb693d-bed4-4e95-89d6-281c802e36b9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:31.001418 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fb693d-bed4-4e95-89d6-281c802e36b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:31.002981 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fb693d-bed4-4e95-89d6-281c802e36b9 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:31.015977 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 479/958] 10.222.0.18 () {56 vars in 1115 bytes} [Sat Aug 6 21:51:30 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:51:31.018567 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af8b55d-c390-4546-b0eb-d07dbc1f4591 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:31.138199 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af8b55d-c390-4546-b0eb-d07dbc1f4591 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:31.175524 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8402a6f1-451e-44f1-8281-7137025cac1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:31.176592 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8402a6f1-451e-44f1-8281-7137025cac1c None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:31.177078 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8402a6f1-451e-44f1-8281-7137025cac1c None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:31.177553 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8402a6f1-451e-44f1-8281-7137025cac1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:31.191023 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8402a6f1-451e-44f1-8281-7137025cac1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:31.192821 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8402a6f1-451e-44f1-8281-7137025cac1c None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:31.207692 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-8402a6f1-451e-44f1-8281-7137025cac1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:51:31.207692 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:31.207692 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:31.207692 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:31.207692 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:31.207692 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:31.207692 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:31.207692 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:31.207692 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:31.207692 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:31.207692 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:31.207692 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:31.207692 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:51:31.207692 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:51:31.207692 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:51:31.207692 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:51:31.208706 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:31.208706 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:31.208706 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:31.208706 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:31.208706 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:31.208706 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:31.208706 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:31.208706 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:31.208706 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:31.208706 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:31.208706 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:31.208706 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:31.208706 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:31.208706 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:31.208706 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:31.208706 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:31.208706 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:51:31.209614 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:51:31.209614 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:51:31.209614 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:31.216132 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 480/959] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:51:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:31.218010 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fb693d-bed4-4e95-89d6-281c802e36b9 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:31.320111 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fb693d-bed4-4e95-89d6-281c802e36b9 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:31.344276 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:31.366824 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dfdbff-b365-4eb3-97d7-ad1e43173eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:31.367764 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dfdbff-b365-4eb3-97d7-ad1e43173eb4 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:31.368239 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dfdbff-b365-4eb3-97d7-ad1e43173eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:31.368765 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dfdbff-b365-4eb3-97d7-ad1e43173eb4 None admin] PATH_INFO: `/v3/domains` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:31.369623 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dfdbff-b365-4eb3-97d7-ad1e43173eb4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:31.370120 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dfdbff-b365-4eb3-97d7-ad1e43173eb4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:31.371706 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dfdbff-b365-4eb3-97d7-ad1e43173eb4 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:31.384861 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 480/960] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:51:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:31.393191 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8402a6f1-451e-44f1-8281-7137025cac1c None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:31.430859 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:31.501109 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8402a6f1-451e-44f1-8281-7137025cac1c None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:31.536051 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963dbbfd-b84f-4ffe-8c25-bd12256ece19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:31.537064 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963dbbfd-b84f-4ffe-8c25-bd12256ece19 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:31.537540 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963dbbfd-b84f-4ffe-8c25-bd12256ece19 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:31.538027 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963dbbfd-b84f-4ffe-8c25-bd12256ece19 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:31.546361 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963dbbfd-b84f-4ffe-8c25-bd12256ece19 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:31.548078 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963dbbfd-b84f-4ffe-8c25-bd12256ece19 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:31.559575 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-963dbbfd-b84f-4ffe-8c25-bd12256ece19 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 06 21:51:31.559575 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:31.559575 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:31.559575 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:31.559575 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:31.559575 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:31.559575 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:31.559575 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:31.559575 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:31.559575 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:31.559575 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:31.559575 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:31.559575 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 21:51:31.559575 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 21:51:31.559575 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 21:51:31.559575 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 21:51:31.560590 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:31.560590 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:31.560590 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 21:51:31.560590 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 21:51:31.560590 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 21:51:31.560590 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 21:51:31.560590 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:31.560590 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:31.560590 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:31.560590 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:31.560590 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:31.560590 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:31.560590 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:31.560590 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:31.560590 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:31.560590 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:31.560590 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:31.560590 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:31.562111 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:31.562111 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:31.562111 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 21:51:31.562111 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 21:51:31.562111 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 21:51:31.562111 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 21:51:31.562111 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 21:51:31.562111 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 21:51:31.562111 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 06 21:51:31.562111 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:31.564335 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 481/961] 10.222.0.18 () {56 vars in 1117 bytes} [Sat Aug 6 21:51:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:31.573382 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dfdbff-b365-4eb3-97d7-ad1e43173eb4 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:31.676790 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dfdbff-b365-4eb3-97d7-ad1e43173eb4 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:31.713987 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1f546a-e54f-47e5-b43f-a120455ce6cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:31.714925 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f546a-e54f-47e5-b43f-a120455ce6cc None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:31.715383 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f546a-e54f-47e5-b43f-a120455ce6cc None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:31.715944 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f546a-e54f-47e5-b43f-a120455ce6cc None admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:31.716927 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1f546a-e54f-47e5-b43f-a120455ce6cc None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:31.717415 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1f546a-e54f-47e5-b43f-a120455ce6cc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:31.719040 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1f546a-e54f-47e5-b43f-a120455ce6cc None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:31.741813 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 481/962] 10.222.0.18 () {56 vars in 1149 bytes} [Sat Aug 6 21:51:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:31.744371 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963dbbfd-b84f-4ffe-8c25-bd12256ece19 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:31.797239 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da670963-1313-458f-993e-89089cf5cdc6 None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:31.797510 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da670963-1313-458f-993e-89089cf5cdc6 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:31.797779 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da670963-1313-458f-993e-89089cf5cdc6 None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:31.799378 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 482/963] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:51:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:51:31.804200 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6cb3ae-ac0c-4f74-9b8f-56585eddca65 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:31.804516 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6cb3ae-ac0c-4f74-9b8f-56585eddca65 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:31.804768 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6cb3ae-ac0c-4f74-9b8f-56585eddca65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:31.823359 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-df6cb3ae-ac0c-4f74-9b8f-56585eddca65 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:51:31.824562 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-df6cb3ae-ac0c-4f74-9b8f-56585eddca65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:51:31.862314 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:31.865899 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963dbbfd-b84f-4ffe-8c25-bd12256ece19 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:31.868710 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:31.904897 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 483/964] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:51:31 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 21:51:31.906905 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddff616e-77f1-44af-92e4-200f8aad535f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:31.907559 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddff616e-77f1-44af-92e4-200f8aad535f None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:31.907890 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddff616e-77f1-44af-92e4-200f8aad535f None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:31.908160 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddff616e-77f1-44af-92e4-200f8aad535f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:31.923651 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6cb3ae-ac0c-4f74-9b8f-56585eddca65 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:31.993982 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddff616e-77f1-44af-92e4-200f8aad535f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:31.995842 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddff616e-77f1-44af-92e4-200f8aad535f None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:32.057861 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6cb3ae-ac0c-4f74-9b8f-56585eddca65 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:32.095753 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:32.097969 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a101d0-0deb-4c89-961a-11876f5c7e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:32.098587 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a101d0-0deb-4c89-961a-11876f5c7e4c None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:32.099060 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a101d0-0deb-4c89-961a-11876f5c7e4c None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:32.099169 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a101d0-0deb-4c89-961a-11876f5c7e4c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:32.099708 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a101d0-0deb-4c89-961a-11876f5c7e4c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:32.101165 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a101d0-0deb-4c89-961a-11876f5c7e4c None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:32.106983 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 482/965] 10.222.0.18 () {58 vars in 1282 bytes} [Sat Aug 6 21:51:31 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 21:51:32.106983 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5663212f-110f-45ae-877e-26826213c44d None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:32.106983 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5663212f-110f-45ae-877e-26826213c44d None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:32.106983 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5663212f-110f-45ae-877e-26826213c44d None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:32.106983 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 483/966] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:51:32 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:51:32.111089 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5663212f-110f-45ae-877e-26826213c44d None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:32.116190 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-55a101d0-0deb-4c89-961a-11876f5c7e4c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 06 21:51:32.116190 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:32.116190 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 21:51:32.116190 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 21:51:32.116190 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 21:51:32.116190 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 21:51:32.116190 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 21:51:32.116190 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 21:51:32.116190 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 06 21:51:32.116190 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:32.116190 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 21:51:32.116190 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:32.116190 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:32.116190 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:32.116190 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:32.116190 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:32.116190 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:32.117482 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:32.117482 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:32.117482 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:32.117482 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:32.117482 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:32.117482 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:32.117482 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 21:51:32.117482 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 21:51:32.117482 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 21:51:32.117482 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 21:51:32.117482 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:32.117482 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:32.117482 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:32.117482 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:32.117482 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:32.117482 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:32.118400 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:32.118400 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:32.118400 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:32.118400 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:32.118400 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:32.118400 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:32.118400 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:32.118400 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:32.118400 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:32.118400 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:32.118400 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 21:51:32.118400 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 21:51:32.118400 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 06 21:51:32.118400 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:32.118400 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 484/967] 10.222.0.18 () {56 vars in 1126 bytes} [Sat Aug 6 21:51:31 2022] GET /identity/v3/services/object-store => generated 93 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:32.119247 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b154b110-17d6-4aea-8b17-8542c198f1f9 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:32.119327 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b154b110-17d6-4aea-8b17-8542c198f1f9 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:32.119868 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b154b110-17d6-4aea-8b17-8542c198f1f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:32.138759 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-b154b110-17d6-4aea-8b17-8542c198f1f9 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:51:32.140050 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-b154b110-17d6-4aea-8b17-8542c198f1f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:51:32.171058 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:32.212603 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 485/968] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:51:32 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 21:51:32.212603 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b154b110-17d6-4aea-8b17-8542c198f1f9 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:32.233454 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5663212f-110f-45ae-877e-26826213c44d None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:32.270257 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6af9b5-7f69-4542-9b24-adb716cb1a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:32.270806 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6af9b5-7f69-4542-9b24-adb716cb1a4b None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:32.271088 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6af9b5-7f69-4542-9b24-adb716cb1a4b None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:32.271356 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6af9b5-7f69-4542-9b24-adb716cb1a4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:32.283128 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6af9b5-7f69-4542-9b24-adb716cb1a4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:32.284633 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6af9b5-7f69-4542-9b24-adb716cb1a4b None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:32.299765 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-bc6af9b5-7f69-4542-9b24-adb716cb1a4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:51:32.299765 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:32.299765 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:32.299765 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:32.299765 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:32.299765 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:32.299765 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:32.299765 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:32.299765 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:32.299765 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:32.299765 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:32.299765 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:32.299765 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:51:32.299765 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:51:32.299765 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:51:32.299765 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:51:32.300915 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:32.300915 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:32.300915 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:32.300915 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:32.300915 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:32.300915 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:32.300915 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:32.300915 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:32.300915 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:32.300915 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:32.300915 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:32.300915 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:32.300915 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:32.300915 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:32.300915 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:32.300915 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:32.300915 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:51:32.301810 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:51:32.301810 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:51:32.301810 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:32.301810 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 484/969] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:51:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:32.301810 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6af9b5-7f69-4542-9b24-adb716cb1a4b None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:32.328595 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b154b110-17d6-4aea-8b17-8542c198f1f9 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:32.355536 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:32.374400 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6f3f9a-cb90-49c1-ba3f-92370e3f99f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:32.375014 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f3f9a-cb90-49c1-ba3f-92370e3f99f5 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:32.375291 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f3f9a-cb90-49c1-ba3f-92370e3f99f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:32.375561 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f3f9a-cb90-49c1-ba3f-92370e3f99f5 None admin] PATH_INFO: `/v3/services` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:32.376238 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6f3f9a-cb90-49c1-ba3f-92370e3f99f5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:32.376599 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6f3f9a-cb90-49c1-ba3f-92370e3f99f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:32.377971 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6f3f9a-cb90-49c1-ba3f-92370e3f99f5 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:32.387530 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 486/970] 10.222.0.18 () {56 vars in 1122 bytes} [Sat Aug 6 21:51:32 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:32.387530 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6f3f9a-cb90-49c1-ba3f-92370e3f99f5 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:32.415908 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6af9b5-7f69-4542-9b24-adb716cb1a4b None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:32.420271 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:32.450973 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6532767e-3233-4cc7-a1a2-fcabd7b1d347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:32.451668 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6532767e-3233-4cc7-a1a2-fcabd7b1d347 None admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:32.451945 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6532767e-3233-4cc7-a1a2-fcabd7b1d347 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:32.452230 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6532767e-3233-4cc7-a1a2-fcabd7b1d347 None admin] PATH_INFO: `/v3/services` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:32.452896 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6532767e-3233-4cc7-a1a2-fcabd7b1d347 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:32.454273 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6532767e-3233-4cc7-a1a2-fcabd7b1d347 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:32.494812 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6f3f9a-cb90-49c1-ba3f-92370e3f99f5 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:32.529368 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40000ca4-5a77-4e48-b16c-6fa3b9e3caa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:32.529894 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40000ca4-5a77-4e48-b16c-6fa3b9e3caa0 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:32.530230 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40000ca4-5a77-4e48-b16c-6fa3b9e3caa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:32.530510 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40000ca4-5a77-4e48-b16c-6fa3b9e3caa0 None admin] PATH_INFO: `/v3/domains` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:32.531158 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40000ca4-5a77-4e48-b16c-6fa3b9e3caa0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:32.531463 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40000ca4-5a77-4e48-b16c-6fa3b9e3caa0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:32.533024 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40000ca4-5a77-4e48-b16c-6fa3b9e3caa0 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:32.540539 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 485/971] 10.222.0.18 () {60 vars in 1141 bytes} [Sat Aug 6 21:51:32 2022] POST /identity/v3/services => generated 242 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 21:51:32.540539 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6532767e-3233-4cc7-a1a2-fcabd7b1d347 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:32.549999 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 487/972] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:51:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:32.551768 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40000ca4-5a77-4e48-b16c-6fa3b9e3caa0 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:32.653658 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:32.663553 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40000ca4-5a77-4e48-b16c-6fa3b9e3caa0 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:32.678886 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6532767e-3233-4cc7-a1a2-fcabd7b1d347 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:32.682014 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:32.706537 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc001116-9e37-4a43-96ea-4f50aed045c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:32.707145 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc001116-9e37-4a43-96ea-4f50aed045c9 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:32.708317 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc001116-9e37-4a43-96ea-4f50aed045c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:32.708317 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc001116-9e37-4a43-96ea-4f50aed045c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:32.715455 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc001116-9e37-4a43-96ea-4f50aed045c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:32.717097 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc001116-9e37-4a43-96ea-4f50aed045c9 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:32.729529 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fb5e9c-ceab-4851-af40-95f1169b3b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:32.730409 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-bc001116-9e37-4a43-96ea-4f50aed045c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 21:51:32.730409 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:32.730409 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:32.730409 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:32.730409 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:32.730409 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:32.730409 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:32.730409 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:32.730409 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:32.730409 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:32.730409 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:32.730409 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:32.730409 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 21:51:32.730409 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 21:51:32.730409 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 21:51:32.731259 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 21:51:32.731259 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:32.731259 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:32.731259 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:32.731259 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:32.731259 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:32.731259 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:32.731259 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:32.731259 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:32.731259 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:32.731259 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:32.731259 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:32.731259 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:32.731259 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:32.731259 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:32.731259 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:32.731259 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:32.732951 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 21:51:32.732951 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 21:51:32.732951 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 21:51:32.732951 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 21:51:32.732951 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 21:51:32.732951 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 21:51:32.732951 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 21:51:32.732951 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:32.732951 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 488/973] 10.222.0.18 () {56 vars in 1111 bytes} [Sat Aug 6 21:51:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 21:51:32.733979 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fb5e9c-ceab-4851-af40-95f1169b3b69 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:32.734422 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fb5e9c-ceab-4851-af40-95f1169b3b69 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:32.734838 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fb5e9c-ceab-4851-af40-95f1169b3b69 None admin] PATH_INFO: `/v3/services` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:32.735628 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fb5e9c-ceab-4851-af40-95f1169b3b69 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:32.736088 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fb5e9c-ceab-4851-af40-95f1169b3b69 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:32.737705 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fb5e9c-ceab-4851-af40-95f1169b3b69 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:32.738146 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc001116-9e37-4a43-96ea-4f50aed045c9 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:32.745240 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 486/974] 10.222.0.18 () {56 vars in 1122 bytes} [Sat Aug 6 21:51:32 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:32.753140 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fb5e9c-ceab-4851-af40-95f1169b3b69 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:32.862167 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc001116-9e37-4a43-96ea-4f50aed045c9 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:32.883892 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fb5e9c-ceab-4851-af40-95f1169b3b69 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:32.899383 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67c33a8-49d9-443b-8d46-10fbf8ddaa7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:32.899989 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67c33a8-49d9-443b-8d46-10fbf8ddaa7d None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:32.900268 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67c33a8-49d9-443b-8d46-10fbf8ddaa7d None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:32.900587 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67c33a8-49d9-443b-8d46-10fbf8ddaa7d None admin] PATH_INFO: `/v3/projects` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:32.901295 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67c33a8-49d9-443b-8d46-10fbf8ddaa7d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:32.901600 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67c33a8-49d9-443b-8d46-10fbf8ddaa7d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:32.903039 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67c33a8-49d9-443b-8d46-10fbf8ddaa7d None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:32.914946 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 489/975] 10.222.0.18 () {56 vars in 1148 bytes} [Sat Aug 6 21:51:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:51:32.916153 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b667ce85-9df3-4843-a82d-d5d74474cb2c None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:32.916903 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b667ce85-9df3-4843-a82d-d5d74474cb2c None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:32.916903 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b667ce85-9df3-4843-a82d-d5d74474cb2c None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:32.918175 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 490/976] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:51:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:51:32.921486 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b667ce85-9df3-4843-a82d-d5d74474cb2c None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:32.925826 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a486c1-eaf2-44b2-958f-8f737f3514d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:32.926642 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a486c1-eaf2-44b2-958f-8f737f3514d0 None admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:32.927068 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a486c1-eaf2-44b2-958f-8f737f3514d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:32.927493 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a486c1-eaf2-44b2-958f-8f737f3514d0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:32.928290 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a486c1-eaf2-44b2-958f-8f737f3514d0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:32.929832 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a486c1-eaf2-44b2-958f-8f737f3514d0 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:32.934187 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:33.025522 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b667ce85-9df3-4843-a82d-d5d74474cb2c None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:33.035877 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 487/977] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 21:51:32 2022] POST /identity/v3/endpoints => generated 349 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 21:51:33.037202 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba837fdc-4c57-4bf0-aad7-8a3f82d40d3f None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:33.037488 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba837fdc-4c57-4bf0-aad7-8a3f82d40d3f None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:33.037764 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba837fdc-4c57-4bf0-aad7-8a3f82d40d3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:33.058078 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-ba837fdc-4c57-4bf0-aad7-8a3f82d40d3f None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:51:33.059334 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-ba837fdc-4c57-4bf0-aad7-8a3f82d40d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:51:33.082343 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:33.091770 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb940375-a3de-4279-a9f2-dab93cdc6f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:33.093599 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb940375-a3de-4279-a9f2-dab93cdc6f3c None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:33.093864 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb940375-a3de-4279-a9f2-dab93cdc6f3c None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:33.094454 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb940375-a3de-4279-a9f2-dab93cdc6f3c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:33.095255 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb940375-a3de-4279-a9f2-dab93cdc6f3c None admin] RBAC: Adding query filter params (role.id=08bf738879a941dab8ac6323043dc991, scope.project.id=4bb6307eb0644d85a1d1daaa61352697, user.id=d1b26ba592724dd389584bb800d877a2) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:33.095563 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb940375-a3de-4279-a9f2-dab93cdc6f3c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:33.097084 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb940375-a3de-4279-a9f2-dab93cdc6f3c None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:33.105658 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 491/978] 10.222.0.18 () {56 vars in 1374 bytes} [Sat Aug 6 21:51:32 2022] GET /identity/v3/role_assignments?user.id=d1b26ba592724dd389584bb800d877a2&scope.project.id=4bb6307eb0644d85a1d1daaa61352697&role.id=08bf738879a941dab8ac6323043dc991 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:33.159566 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 488/979] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:51:33 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 21:51:33.188107 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb940375-a3de-4279-a9f2-dab93cdc6f3c None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:33.269536 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:33.304142 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb940375-a3de-4279-a9f2-dab93cdc6f3c None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:33.343972 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a437f007-7ea8-4fde-8612-e2d78787d802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:33.344626 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a437f007-7ea8-4fde-8612-e2d78787d802 None admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:33.344906 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a437f007-7ea8-4fde-8612-e2d78787d802 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:33.345178 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a437f007-7ea8-4fde-8612-e2d78787d802 None admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:33.345838 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a437f007-7ea8-4fde-8612-e2d78787d802 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:33.347275 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a437f007-7ea8-4fde-8612-e2d78787d802 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:33.412096 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.sql.core [None req-a437f007-7ea8-4fde-8612-e2d78787d802 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 06 21:51:33.412096 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 06 21:51:33.412096 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [parameters: {'user_id': 'cdcde4da1e13438abfb6e3d26030482b', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 06 21:51:33.412096 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80423) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 06 21:51:33.419590 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-a437f007-7ea8-4fde-8612-e2d78787d802 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 06 21:51:33.419590 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:33.419590 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 06 21:51:33.419590 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 06 21:51:33.419590 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 06 21:51:33.419590 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 06 21:51:33.419590 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 06 21:51:33.419590 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application result = self._query(query) Aug 06 21:51:33.419590 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 06 21:51:33.419590 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application conn.query(q) Aug 06 21:51:33.419590 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 06 21:51:33.419590 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 06 21:51:33.419590 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 06 21:51:33.419590 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application result.read() Aug 06 21:51:33.419590 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 06 21:51:33.421340 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 06 21:51:33.421340 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 06 21:51:33.421340 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 06 21:51:33.421340 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 06 21:51:33.421340 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 06 21:51:33.421340 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 06 21:51:33.421340 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 06 21:51:33.421340 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 06 21:51:33.421340 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:33.421340 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 06 21:51:33.421340 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:33.421340 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:33.421340 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 06 21:51:33.421340 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 06 21:51:33.421340 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 06 21:51:33.421340 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 06 21:51:33.421340 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 06 21:51:33.421340 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application next(self.gen) Aug 06 21:51:33.422312 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 06 21:51:33.422312 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application yield resource Aug 06 21:51:33.422312 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 06 21:51:33.422312 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application next(self.gen) Aug 06 21:51:33.422312 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 06 21:51:33.422312 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application self.session.rollback() Aug 06 21:51:33.422312 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 06 21:51:33.422312 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application self.force_reraise() Aug 06 21:51:33.422312 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 06 21:51:33.422312 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise self.value Aug 06 21:51:33.422312 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 06 21:51:33.422312 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 06 21:51:33.422312 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 06 21:51:33.422312 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application session.commit() Aug 06 21:51:33.422312 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 06 21:51:33.422312 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 06 21:51:33.422312 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 06 21:51:33.423325 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application self._prepare_impl() Aug 06 21:51:33.423325 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 06 21:51:33.423325 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application self.session.flush() Aug 06 21:51:33.423325 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 06 21:51:33.423325 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application self._flush(objects) Aug 06 21:51:33.423325 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 06 21:51:33.423325 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 06 21:51:33.423325 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 06 21:51:33.423325 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application compat.raise_( Aug 06 21:51:33.423325 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 06 21:51:33.423325 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception Aug 06 21:51:33.423325 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 06 21:51:33.423325 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application flush_context.execute() Aug 06 21:51:33.423325 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 06 21:51:33.423325 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rec.execute(self) Aug 06 21:51:33.423325 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 06 21:51:33.423325 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 06 21:51:33.424205 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 06 21:51:33.424205 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 06 21:51:33.424205 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 06 21:51:33.424205 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 06 21:51:33.424205 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 06 21:51:33.424205 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 06 21:51:33.424205 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 06 21:51:33.424205 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 06 21:51:33.424205 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 06 21:51:33.424205 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 06 21:51:33.424205 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 06 21:51:33.424205 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 06 21:51:33.424205 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 06 21:51:33.424205 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 06 21:51:33.424205 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 06 21:51:33.424205 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception Aug 06 21:51:33.426749 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 06 21:51:33.426749 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 06 21:51:33.426749 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 06 21:51:33.426749 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 06 21:51:33.426749 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 06 21:51:33.426749 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application result = self._query(query) Aug 06 21:51:33.426749 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 06 21:51:33.426749 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application conn.query(q) Aug 06 21:51:33.426749 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 06 21:51:33.426749 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 06 21:51:33.426749 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 06 21:51:33.426749 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application result.read() Aug 06 21:51:33.426749 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 06 21:51:33.426749 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 06 21:51:33.426749 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 06 21:51:33.426749 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 06 21:51:33.428866 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 06 21:51:33.428866 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 06 21:51:33.428866 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 06 21:51:33.428866 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 06 21:51:33.428866 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 06 21:51:33.428866 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 06 21:51:33.428866 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [parameters: {'user_id': 'cdcde4da1e13438abfb6e3d26030482b', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 06 21:51:33.428866 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 06 21:51:33.428866 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:33.428866 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 21:51:33.428866 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:33.428866 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:33.428866 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:33.428866 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:33.428866 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:33.430535 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:33.430535 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:33.430535 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:33.430535 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:33.430535 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:33.430535 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:33.430535 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:33.430535 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 06 21:51:33.430535 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 06 21:51:33.430535 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:33.430535 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:33.430535 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 21:51:33.430535 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 21:51:33.430535 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 21:51:33.430535 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 21:51:33.430535 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 06 21:51:33.430535 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 06 21:51:33.432110 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 06 21:51:33.432110 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 06 21:51:33.432110 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 06 21:51:33.432110 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 06 21:51:33.432110 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 06 21:51:33.432110 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:33.432110 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 492/980] 10.222.0.18 () {60 vars in 1131 bytes} [Sat Aug 6 21:51:33 2022] POST /identity/v3/users => generated 160 bytes in 234 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 06 21:51:33.439063 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba837fdc-4c57-4bf0-aad7-8a3f82d40d3f None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:33.535850 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba837fdc-4c57-4bf0-aad7-8a3f82d40d3f None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:33.571233 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60eaa08-2462-404f-9004-48188681758d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:33.571865 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60eaa08-2462-404f-9004-48188681758d None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:33.572138 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60eaa08-2462-404f-9004-48188681758d None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:33.572423 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60eaa08-2462-404f-9004-48188681758d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:33.579861 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60eaa08-2462-404f-9004-48188681758d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:33.581339 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60eaa08-2462-404f-9004-48188681758d None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:33.593604 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-d60eaa08-2462-404f-9004-48188681758d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 06 21:51:33.593604 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:33.593604 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:33.593604 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:33.593604 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:33.593604 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:33.593604 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:33.593604 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:33.593604 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:33.593604 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:33.593604 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:33.593604 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:33.593604 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 21:51:33.593604 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 21:51:33.593604 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 21:51:33.593604 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 21:51:33.594587 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:33.594587 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:33.594587 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 21:51:33.594587 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 21:51:33.594587 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 21:51:33.594587 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 21:51:33.594587 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:33.594587 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:33.594587 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:33.594587 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:33.594587 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:33.594587 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:33.594587 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:33.594587 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:33.594587 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:33.594587 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:33.594587 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:33.594587 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:33.595862 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:33.595862 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:33.595862 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 21:51:33.595862 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 21:51:33.595862 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 21:51:33.595862 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 21:51:33.595862 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 21:51:33.595862 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 21:51:33.595862 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 06 21:51:33.595862 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:33.595862 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 489/981] 10.222.0.18 () {56 vars in 1099 bytes} [Sat Aug 6 21:51:33 2022] GET /identity/v3/users/cinder => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:33.600756 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a437f007-7ea8-4fde-8612-e2d78787d802 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:33.714611 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a437f007-7ea8-4fde-8612-e2d78787d802 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:33.745953 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:33.751997 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c68b2ff-7de9-4d19-9616-55f8e525bf98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:33.752723 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c68b2ff-7de9-4d19-9616-55f8e525bf98 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:33.753010 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c68b2ff-7de9-4d19-9616-55f8e525bf98 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:33.753284 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c68b2ff-7de9-4d19-9616-55f8e525bf98 None admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:33.753996 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c68b2ff-7de9-4d19-9616-55f8e525bf98 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:33.754298 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c68b2ff-7de9-4d19-9616-55f8e525bf98 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:33.758805 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c68b2ff-7de9-4d19-9616-55f8e525bf98 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:33.798077 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 493/982] 10.222.0.18 () {56 vars in 1101 bytes} [Sat Aug 6 21:51:33 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:51:34.711698 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adbf1c0-f3f3-4ca4-9e2b-b797cf74d114 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:34.712231 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adbf1c0-f3f3-4ca4-9e2b-b797cf74d114 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:34.712231 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adbf1c0-f3f3-4ca4-9e2b-b797cf74d114 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:34.713651 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 490/983] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:51:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:51:34.719171 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d6e7f-9070-482c-ac10-a652b933da68 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:34.719586 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d6e7f-9070-482c-ac10-a652b933da68 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:34.719770 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d6e7f-9070-482c-ac10-a652b933da68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:34.740386 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-f50d6e7f-9070-482c-ac10-a652b933da68 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:51:34.741925 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-f50d6e7f-9070-482c-ac10-a652b933da68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:51:34.812179 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 494/984] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:51:34 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 21:51:34.834780 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adbf1c0-f3f3-4ca4-9e2b-b797cf74d114 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:34.852042 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:34.939957 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adbf1c0-f3f3-4ca4-9e2b-b797cf74d114 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:34.974430 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a72543-4600-4478-b6fc-ad01f5f2ea5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:34.974977 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a72543-4600-4478-b6fc-ad01f5f2ea5b None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:34.975256 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a72543-4600-4478-b6fc-ad01f5f2ea5b None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:34.975527 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a72543-4600-4478-b6fc-ad01f5f2ea5b None admin] PATH_INFO: `/v3/services/compute` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:34.976103 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a72543-4600-4478-b6fc-ad01f5f2ea5b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:34.977505 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a72543-4600-4478-b6fc-ad01f5f2ea5b None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:34.986836 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-02a72543-4600-4478-b6fc-ad01f5f2ea5b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 06 21:51:34.986836 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:34.986836 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 21:51:34.986836 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 21:51:34.986836 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 21:51:34.986836 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 21:51:34.986836 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 21:51:34.986836 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 21:51:34.986836 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 06 21:51:34.986836 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:34.986836 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 21:51:34.986836 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:34.986836 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:34.986836 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:34.986836 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:34.986836 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:34.986836 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:34.987848 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:34.987848 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:34.987848 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:34.987848 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:34.987848 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:34.987848 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:34.987848 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 21:51:34.987848 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 21:51:34.987848 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 21:51:34.987848 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 21:51:34.987848 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:34.987848 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:34.987848 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:34.987848 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:34.987848 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:34.987848 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:34.988751 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:34.988751 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:34.988751 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:34.988751 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:34.988751 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:34.988751 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:34.988751 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:34.988751 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:34.988751 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:34.988751 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:34.988751 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 21:51:34.988751 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 21:51:34.988751 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 06 21:51:34.988751 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:34.992466 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 491/985] 10.222.0.18 () {56 vars in 1111 bytes} [Sat Aug 6 21:51:34 2022] GET /identity/v3/services/compute => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:34.994230 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50d6e7f-9070-482c-ac10-a652b933da68 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:35.095910 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50d6e7f-9070-482c-ac10-a652b933da68 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:35.117509 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:35.132609 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c21e42-d18f-4dce-82e1-9e881d0ff9c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:35.133221 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c21e42-d18f-4dce-82e1-9e881d0ff9c8 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:35.133537 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c21e42-d18f-4dce-82e1-9e881d0ff9c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:35.133841 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c21e42-d18f-4dce-82e1-9e881d0ff9c8 None admin] PATH_INFO: `/v3/services` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:35.134474 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c21e42-d18f-4dce-82e1-9e881d0ff9c8 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:35.134779 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c21e42-d18f-4dce-82e1-9e881d0ff9c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:35.136310 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c21e42-d18f-4dce-82e1-9e881d0ff9c8 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:35.145943 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 495/986] 10.222.0.18 () {56 vars in 1112 bytes} [Sat Aug 6 21:51:34 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:51:35.147753 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a72543-4600-4478-b6fc-ad01f5f2ea5b None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:35.196419 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b893dcac-8dc8-46a4-b721-27132ee3804c None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:35.196773 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b893dcac-8dc8-46a4-b721-27132ee3804c None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:35.196974 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b893dcac-8dc8-46a4-b721-27132ee3804c None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:35.198499 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 496/987] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:51:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 21:51:35.203552 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264effb5-6fa0-4fb2-ba0a-d5f251412e57 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:35.203836 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264effb5-6fa0-4fb2-ba0a-d5f251412e57 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:35.204114 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264effb5-6fa0-4fb2-ba0a-d5f251412e57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:35.222787 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-264effb5-6fa0-4fb2-ba0a-d5f251412e57 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:51:35.223965 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-264effb5-6fa0-4fb2-ba0a-d5f251412e57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:51:35.231535 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:35.254634 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a72543-4600-4478-b6fc-ad01f5f2ea5b None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:35.290106 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c368ad23-56d3-4ac1-9a8a-cd5f3d65faf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:35.290710 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c368ad23-56d3-4ac1-9a8a-cd5f3d65faf9 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:35.290990 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c368ad23-56d3-4ac1-9a8a-cd5f3d65faf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:35.291263 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c368ad23-56d3-4ac1-9a8a-cd5f3d65faf9 None admin] PATH_INFO: `/v3/services` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:35.291893 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c368ad23-56d3-4ac1-9a8a-cd5f3d65faf9 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:35.292196 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c368ad23-56d3-4ac1-9a8a-cd5f3d65faf9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:35.296669 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 497/988] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:51:35 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 21:51:35.296669 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352d786f-cb4f-437c-8152-0f21434eb58b None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:35.296889 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c368ad23-56d3-4ac1-9a8a-cd5f3d65faf9 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:35.296969 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352d786f-cb4f-437c-8152-0f21434eb58b None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:35.296969 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352d786f-cb4f-437c-8152-0f21434eb58b None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:35.296969 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 498/989] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:51:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:51:35.300684 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 492/990] 10.222.0.18 () {56 vars in 1112 bytes} [Sat Aug 6 21:51:35 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:35.301371 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c394ddb6-54fd-4881-a99a-820e4224f8f3 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:35.301649 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c394ddb6-54fd-4881-a99a-820e4224f8f3 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:35.301915 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c394ddb6-54fd-4881-a99a-820e4224f8f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:35.306251 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c368ad23-56d3-4ac1-9a8a-cd5f3d65faf9 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:35.320100 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-c394ddb6-54fd-4881-a99a-820e4224f8f3 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:51:35.321296 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-c394ddb6-54fd-4881-a99a-820e4224f8f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:51:35.393109 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 499/991] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:51:35 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 21:51:35.393956 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c394ddb6-54fd-4881-a99a-820e4224f8f3 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:35.404384 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c368ad23-56d3-4ac1-9a8a-cd5f3d65faf9 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:35.438108 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3bc0d7-d9db-4a45-ba46-81e0221e2b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:35.438715 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3bc0d7-d9db-4a45-ba46-81e0221e2b2e None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:35.438995 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3bc0d7-d9db-4a45-ba46-81e0221e2b2e None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:35.439266 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3bc0d7-d9db-4a45-ba46-81e0221e2b2e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:35.439947 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:35.440039 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3bc0d7-d9db-4a45-ba46-81e0221e2b2e None admin] RBAC: Adding query filter params (interface=public, service_id=e81be2594e084b3489d5c261d0a8b02a, region_id=RegionOne) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:35.440245 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3bc0d7-d9db-4a45-ba46-81e0221e2b2e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:35.441647 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3bc0d7-d9db-4a45-ba46-81e0221e2b2e None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:35.446608 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 493/992] 10.222.0.18 () {56 vars in 1251 bytes} [Sat Aug 6 21:51:35 2022] GET /identity/v3/endpoints?service_id=e81be2594e084b3489d5c261d0a8b02a&interface=public®ion_id=RegionOne => generated 194 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:51:35.447389 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3bc0d7-d9db-4a45-ba46-81e0221e2b2e None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:35.459742 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:35.495628 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c394ddb6-54fd-4881-a99a-820e4224f8f3 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:35.529050 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a76350-5f26-4c64-a832-ebb7256abfbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:35.530247 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a76350-5f26-4c64-a832-ebb7256abfbb None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:35.530530 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a76350-5f26-4c64-a832-ebb7256abfbb None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:35.530798 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a76350-5f26-4c64-a832-ebb7256abfbb None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:35.531385 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a76350-5f26-4c64-a832-ebb7256abfbb None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:35.532798 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a76350-5f26-4c64-a832-ebb7256abfbb None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:35.541330 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-f4a76350-5f26-4c64-a832-ebb7256abfbb None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 06 21:51:35.541330 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:35.541330 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 21:51:35.541330 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 21:51:35.541330 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 21:51:35.541330 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 21:51:35.541330 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 21:51:35.541330 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 21:51:35.541330 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 06 21:51:35.541330 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:35.541330 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 21:51:35.541330 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:35.541330 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:35.541330 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:35.541330 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:35.541330 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:35.541330 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:35.542394 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:35.542394 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:35.542394 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:35.542394 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:35.542394 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:35.542394 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:35.542394 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 21:51:35.542394 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 21:51:35.542394 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 21:51:35.542394 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 21:51:35.542394 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:35.542394 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:35.542394 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:35.542394 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:35.542394 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:35.542394 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:35.543337 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:35.543337 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:35.543337 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:35.543337 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:35.543337 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:35.543337 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:35.543337 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:35.543337 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:35.543337 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:35.543337 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:35.543337 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 21:51:35.543337 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 21:51:35.543337 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 06 21:51:35.543337 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:35.543337 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 500/993] 10.222.0.18 () {56 vars in 1126 bytes} [Sat Aug 6 21:51:35 2022] GET /identity/v3/services/object-store => generated 93 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:35.544294 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a76350-5f26-4c64-a832-ebb7256abfbb None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:35.554299 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3bc0d7-d9db-4a45-ba46-81e0221e2b2e None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:35.588353 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d5a4a7-d887-4f96-98bc-0a264660cb2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:35.588996 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d5a4a7-d887-4f96-98bc-0a264660cb2a None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:35.589271 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d5a4a7-d887-4f96-98bc-0a264660cb2a None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:35.589533 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d5a4a7-d887-4f96-98bc-0a264660cb2a None admin] PATH_INFO: `/v3/services/image` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:35.590100 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d5a4a7-d887-4f96-98bc-0a264660cb2a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:35.591461 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d5a4a7-d887-4f96-98bc-0a264660cb2a None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:35.600597 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-d5d5a4a7-d887-4f96-98bc-0a264660cb2a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 06 21:51:35.600597 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:35.600597 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 21:51:35.600597 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 21:51:35.600597 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 21:51:35.600597 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 21:51:35.600597 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 21:51:35.600597 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 21:51:35.600597 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 06 21:51:35.600597 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:35.600597 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 21:51:35.600597 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:35.600597 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:35.600597 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:35.600597 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:35.600597 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:35.600597 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:35.601591 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:35.601591 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:35.601591 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:35.601591 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:35.601591 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:35.601591 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:35.601591 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 21:51:35.601591 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 21:51:35.601591 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 21:51:35.601591 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 21:51:35.601591 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:35.601591 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:35.601591 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:35.601591 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:35.601591 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:35.601591 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:35.602559 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:35.602559 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:35.602559 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:35.602559 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:35.602559 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:35.602559 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:35.602559 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:35.602559 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:35.602559 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:35.602559 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:35.602559 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 21:51:35.602559 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 21:51:35.602559 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 06 21:51:35.602559 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:35.602559 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 494/994] 10.222.0.18 () {56 vars in 1105 bytes} [Sat Aug 6 21:51:35 2022] GET /identity/v3/services/image => generated 86 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:35.603368 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d5a4a7-d887-4f96-98bc-0a264660cb2a None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:35.647341 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a76350-5f26-4c64-a832-ebb7256abfbb None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:35.656930 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:35.670130 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:35.680446 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778cadd7-dc3d-4009-aa83-3dc9d1fa06b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:35.681049 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778cadd7-dc3d-4009-aa83-3dc9d1fa06b2 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:35.681323 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778cadd7-dc3d-4009-aa83-3dc9d1fa06b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:35.682409 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778cadd7-dc3d-4009-aa83-3dc9d1fa06b2 None admin] PATH_INFO: `/v3/services` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:35.682409 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778cadd7-dc3d-4009-aa83-3dc9d1fa06b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:35.683516 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778cadd7-dc3d-4009-aa83-3dc9d1fa06b2 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:35.689675 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 501/995] 10.222.0.18 () {56 vars in 1087 bytes} [Sat Aug 6 21:51:35 2022] GET /identity/v3/services => generated 1727 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 21:51:35.690454 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778cadd7-dc3d-4009-aa83-3dc9d1fa06b2 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:35.702840 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d5a4a7-d887-4f96-98bc-0a264660cb2a None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:35.742509 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348045cb-b3c8-40af-83b9-aea6d698475e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:35.743178 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348045cb-b3c8-40af-83b9-aea6d698475e None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:35.743390 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348045cb-b3c8-40af-83b9-aea6d698475e None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:35.743658 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348045cb-b3c8-40af-83b9-aea6d698475e None admin] PATH_INFO: `/v3/services` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:35.744302 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348045cb-b3c8-40af-83b9-aea6d698475e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:35.744669 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348045cb-b3c8-40af-83b9-aea6d698475e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:35.746168 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348045cb-b3c8-40af-83b9-aea6d698475e None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:35.756529 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 495/996] 10.222.0.18 () {56 vars in 1122 bytes} [Sat Aug 6 21:51:35 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:35.757743 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348045cb-b3c8-40af-83b9-aea6d698475e None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:35.819343 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778cadd7-dc3d-4009-aa83-3dc9d1fa06b2 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:35.859664 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a680d2-7d35-4625-b62c-d6d51c1d21d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:35.860291 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a680d2-7d35-4625-b62c-d6d51c1d21d2 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:35.860598 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a680d2-7d35-4625-b62c-d6d51c1d21d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:35.860879 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a680d2-7d35-4625-b62c-d6d51c1d21d2 None admin] PATH_INFO: `/v3/services` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:35.861540 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a680d2-7d35-4625-b62c-d6d51c1d21d2 None admin] RBAC: Adding query filter params (name=image) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:35.861844 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a680d2-7d35-4625-b62c-d6d51c1d21d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:35.863224 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a680d2-7d35-4625-b62c-d6d51c1d21d2 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:35.868897 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 502/997] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:51:35 2022] GET /identity/v3/services?name=image => generated 122 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:35.874430 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a680d2-7d35-4625-b62c-d6d51c1d21d2 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:35.878593 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348045cb-b3c8-40af-83b9-aea6d698475e None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:35.912256 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:35.913682 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7099c2-eeb6-4959-bdbe-7ff48c0d7ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:35.914291 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7099c2-eeb6-4959-bdbe-7ff48c0d7ff0 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:35.914570 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7099c2-eeb6-4959-bdbe-7ff48c0d7ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:35.914839 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7099c2-eeb6-4959-bdbe-7ff48c0d7ff0 None admin] PATH_INFO: `/v3/services` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:35.915470 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7099c2-eeb6-4959-bdbe-7ff48c0d7ff0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:35.915771 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7099c2-eeb6-4959-bdbe-7ff48c0d7ff0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:35.917179 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7099c2-eeb6-4959-bdbe-7ff48c0d7ff0 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:35.918600 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:35.925247 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 496/998] 10.222.0.18 () {56 vars in 1122 bytes} [Sat Aug 6 21:51:35 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:35.930315 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7099c2-eeb6-4959-bdbe-7ff48c0d7ff0 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:35.981174 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a680d2-7d35-4625-b62c-d6d51c1d21d2 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:36.020110 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3ed8a4-ffcf-4f57-b4f6-ca0c094389d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:36.020663 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3ed8a4-ffcf-4f57-b4f6-ca0c094389d1 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:36.020944 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3ed8a4-ffcf-4f57-b4f6-ca0c094389d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:36.021220 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3ed8a4-ffcf-4f57-b4f6-ca0c094389d1 None admin] PATH_INFO: `/v3/services` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:36.021933 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3ed8a4-ffcf-4f57-b4f6-ca0c094389d1 None admin] RBAC: Adding query filter params (type=image) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:36.022363 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3ed8a4-ffcf-4f57-b4f6-ca0c094389d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:36.023721 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3ed8a4-ffcf-4f57-b4f6-ca0c094389d1 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:36.028802 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 503/999] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:51:35 2022] GET /identity/v3/services?type=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:51:36.047967 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7099c2-eeb6-4959-bdbe-7ff48c0d7ff0 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:36.087431 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3680bbf-d86d-4382-b902-720dfd70cd0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:36.088363 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3680bbf-d86d-4382-b902-720dfd70cd0b None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:36.088852 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3680bbf-d86d-4382-b902-720dfd70cd0b None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:36.089304 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3680bbf-d86d-4382-b902-720dfd70cd0b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:36.090225 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3680bbf-d86d-4382-b902-720dfd70cd0b None admin] RBAC: Adding query filter params (interface=admin, service_id=557ace9e3e3543aaada4d3cfe53af3b2, region_id=RegionOne) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:36.090708 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3680bbf-d86d-4382-b902-720dfd70cd0b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:36.092627 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3680bbf-d86d-4382-b902-720dfd70cd0b None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:36.099081 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 497/1000] 10.222.0.18 () {56 vars in 1249 bytes} [Sat Aug 6 21:51:35 2022] GET /identity/v3/endpoints?service_id=557ace9e3e3543aaada4d3cfe53af3b2&interface=admin®ion_id=RegionOne => generated 193 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:51:36.104488 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3ed8a4-ffcf-4f57-b4f6-ca0c094389d1 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:36.213697 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:36.217335 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3ed8a4-ffcf-4f57-b4f6-ca0c094389d1 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:36.253240 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e68e400-b6c8-4322-be60-b0b7b489b108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:36.253854 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e68e400-b6c8-4322-be60-b0b7b489b108 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:36.254134 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e68e400-b6c8-4322-be60-b0b7b489b108 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:36.254410 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e68e400-b6c8-4322-be60-b0b7b489b108 None admin] PATH_INFO: `/v3/services` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:36.255119 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e68e400-b6c8-4322-be60-b0b7b489b108 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:36.256384 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e68e400-b6c8-4322-be60-b0b7b489b108 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:36.268444 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 504/1001] 10.222.0.18 () {56 vars in 1087 bytes} [Sat Aug 6 21:51:36 2022] GET /identity/v3/services => generated 1727 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 21:51:37.706921 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b127c7d4-ec63-4163-9fc4-9a1f46819687 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:37.707440 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b127c7d4-ec63-4163-9fc4-9a1f46819687 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:37.707440 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b127c7d4-ec63-4163-9fc4-9a1f46819687 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:37.708953 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 498/1002] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:51:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:51:37.713843 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90d5547-a6ab-44d1-b383-f28e487054e6 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:37.714286 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90d5547-a6ab-44d1-b383-f28e487054e6 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:37.714459 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90d5547-a6ab-44d1-b383-f28e487054e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:37.733113 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-c90d5547-a6ab-44d1-b383-f28e487054e6 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:51:37.734375 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-c90d5547-a6ab-44d1-b383-f28e487054e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:51:37.804560 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 505/1003] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:51:37 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 21:51:37.824041 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b127c7d4-ec63-4163-9fc4-9a1f46819687 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:37.887203 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:37.927203 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b127c7d4-ec63-4163-9fc4-9a1f46819687 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:37.961444 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeb1da4-734a-46f8-b464-a4765b2ec370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:37.961992 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeb1da4-734a-46f8-b464-a4765b2ec370 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:37.962267 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeb1da4-734a-46f8-b464-a4765b2ec370 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:37.962533 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeb1da4-734a-46f8-b464-a4765b2ec370 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:37.963103 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeeb1da4-734a-46f8-b464-a4765b2ec370 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:37.964491 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeeb1da4-734a-46f8-b464-a4765b2ec370 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:37.973309 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-aeeb1da4-734a-46f8-b464-a4765b2ec370 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 06 21:51:37.973309 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:37.973309 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 21:51:37.973309 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 21:51:37.973309 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 21:51:37.973309 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 21:51:37.973309 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 21:51:37.973309 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 21:51:37.973309 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 06 21:51:37.973309 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:37.973309 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 21:51:37.973309 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:37.973309 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:37.973309 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:37.973309 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:37.973309 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:37.973309 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:37.974322 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:37.974322 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:37.974322 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:37.974322 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:37.974322 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:37.974322 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:37.974322 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 21:51:37.974322 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 21:51:37.974322 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 21:51:37.974322 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 21:51:37.974322 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:37.974322 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:37.974322 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:37.974322 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:37.974322 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:37.974322 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:37.980116 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:37.980116 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:37.980116 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:37.980116 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:37.980116 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:37.980116 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:37.980116 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:37.980116 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:37.980116 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:37.980116 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:37.980116 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 21:51:37.980116 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 21:51:37.980116 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 06 21:51:37.980116 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:37.980116 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 499/1004] 10.222.0.18 () {56 vars in 1111 bytes} [Sat Aug 6 21:51:37 2022] GET /identity/v3/services/compute => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:37.981987 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90d5547-a6ab-44d1-b383-f28e487054e6 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:38.078000 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b7809e-64d9-4477-8a0a-441c364dce3d None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:38.078365 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b7809e-64d9-4477-8a0a-441c364dce3d None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:38.078726 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b7809e-64d9-4477-8a0a-441c364dce3d None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:38.079063 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:38.080178 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 500/1005] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:51:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:51:38.085202 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fd2196-1eb5-4b22-abeb-814dfc48b5db None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:38.085496 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fd2196-1eb5-4b22-abeb-814dfc48b5db None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:38.085770 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fd2196-1eb5-4b22-abeb-814dfc48b5db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:38.086153 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90d5547-a6ab-44d1-b383-f28e487054e6 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:38.108559 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-86fd2196-1eb5-4b22-abeb-814dfc48b5db None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:51:38.109813 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-86fd2196-1eb5-4b22-abeb-814dfc48b5db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:51:38.124538 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcbf3d4-8680-4978-8fc7-d2eade588bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:38.125422 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcbf3d4-8680-4978-8fc7-d2eade588bc2 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:38.125893 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcbf3d4-8680-4978-8fc7-d2eade588bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:38.126164 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcbf3d4-8680-4978-8fc7-d2eade588bc2 None admin] PATH_INFO: `/v3/services` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:38.126797 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcbf3d4-8680-4978-8fc7-d2eade588bc2 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:38.127099 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcbf3d4-8680-4978-8fc7-d2eade588bc2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:38.128498 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcbf3d4-8680-4978-8fc7-d2eade588bc2 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:38.133711 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 506/1006] 10.222.0.18 () {56 vars in 1112 bytes} [Sat Aug 6 21:51:37 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:38.139305 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcbf3d4-8680-4978-8fc7-d2eade588bc2 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:38.192696 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 501/1007] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:51:38 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 21:51:38.207576 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fd2196-1eb5-4b22-abeb-814dfc48b5db None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:38.251893 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcbf3d4-8680-4978-8fc7-d2eade588bc2 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:38.267259 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:38.288154 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bb9b0f-ad22-4792-bf78-f5059d96964d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:38.288800 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb9b0f-ad22-4792-bf78-f5059d96964d None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:38.289081 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb9b0f-ad22-4792-bf78-f5059d96964d None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:38.289467 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb9b0f-ad22-4792-bf78-f5059d96964d None admin] PATH_INFO: `/v3/services` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:38.290112 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bb9b0f-ad22-4792-bf78-f5059d96964d None admin] RBAC: Adding query filter params (type=compute) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:38.290576 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bb9b0f-ad22-4792-bf78-f5059d96964d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:38.291946 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bb9b0f-ad22-4792-bf78-f5059d96964d None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:38.297109 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 507/1008] 10.222.0.18 () {56 vars in 1112 bytes} [Sat Aug 6 21:51:38 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:51:38.298266 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc63337-a52a-4a8f-ac26-9c65f998bcaa None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:38.298553 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc63337-a52a-4a8f-ac26-9c65f998bcaa None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:38.298830 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc63337-a52a-4a8f-ac26-9c65f998bcaa None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:38.300026 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 508/1009] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:51:38 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:51:38.302741 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc63337-a52a-4a8f-ac26-9c65f998bcaa None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:38.316039 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fd2196-1eb5-4b22-abeb-814dfc48b5db None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:38.328184 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:38.351749 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80924fe5-629b-464c-b98b-959faf6abcac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:38.352108 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80924fe5-629b-464c-b98b-959faf6abcac None admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:38.352384 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80924fe5-629b-464c-b98b-959faf6abcac None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:38.352686 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80924fe5-629b-464c-b98b-959faf6abcac None admin] PATH_INFO: `/v3/services` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:38.353324 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80924fe5-629b-464c-b98b-959faf6abcac None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:38.354780 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80924fe5-629b-464c-b98b-959faf6abcac None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:38.406231 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc63337-a52a-4a8f-ac26-9c65f998bcaa None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:38.439480 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52246a9-c1ba-4bda-93c0-05dc6e47f2b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:38.440076 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52246a9-c1ba-4bda-93c0-05dc6e47f2b7 None admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:38.440356 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52246a9-c1ba-4bda-93c0-05dc6e47f2b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:38.440659 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52246a9-c1ba-4bda-93c0-05dc6e47f2b7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:38.441306 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52246a9-c1ba-4bda-93c0-05dc6e47f2b7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:38.442658 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52246a9-c1ba-4bda-93c0-05dc6e47f2b7 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:38.458045 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 502/1010] 10.222.0.18 () {60 vars in 1141 bytes} [Sat Aug 6 21:51:38 2022] POST /identity/v3/services => generated 242 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 21:51:38.459274 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f3dbbb-1b4d-4ae9-8a4f-8acfdaf39563 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:38.459553 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f3dbbb-1b4d-4ae9-8a4f-8acfdaf39563 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:38.459820 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f3dbbb-1b4d-4ae9-8a4f-8acfdaf39563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:38.480013 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-91f3dbbb-1b4d-4ae9-8a4f-8acfdaf39563 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:51:38.481274 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-91f3dbbb-1b4d-4ae9-8a4f-8acfdaf39563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:51:38.536524 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 509/1011] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 21:51:38 2022] POST /identity/v3/endpoints => generated 334 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 21:51:38.566808 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 503/1012] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:51:38 2022] POST /identity/v3/auth/tokens => generated 1822 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 21:51:38.589878 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52246a9-c1ba-4bda-93c0-05dc6e47f2b7 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:38.682755 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:38.706201 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52246a9-c1ba-4bda-93c0-05dc6e47f2b7 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:38.742628 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44b51d4-ef10-464b-a279-f209d567dfb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:38.744096 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44b51d4-ef10-464b-a279-f209d567dfb2 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:38.745121 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44b51d4-ef10-464b-a279-f209d567dfb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:38.745121 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44b51d4-ef10-464b-a279-f209d567dfb2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:38.745559 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44b51d4-ef10-464b-a279-f209d567dfb2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:38.747564 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44b51d4-ef10-464b-a279-f209d567dfb2 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:38.757275 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-b44b51d4-ef10-464b-a279-f209d567dfb2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 06 21:51:38.757275 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:38.757275 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 21:51:38.757275 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 21:51:38.757275 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 21:51:38.757275 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 21:51:38.757275 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 21:51:38.757275 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 21:51:38.757275 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 06 21:51:38.757275 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:38.757275 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 21:51:38.757275 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:38.757275 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:38.757275 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:38.757275 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:38.757275 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:38.757275 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:38.758279 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:38.758279 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:38.758279 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:38.758279 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:38.758279 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:38.758279 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:38.758279 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 21:51:38.758279 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 21:51:38.758279 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 21:51:38.758279 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 21:51:38.758279 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:38.758279 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:38.758279 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:38.758279 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:38.758279 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:38.758279 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:38.759114 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:38.759114 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:38.759114 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:38.759114 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:38.759114 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:38.759114 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:38.759114 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:38.759114 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:38.759114 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:38.759114 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:38.759114 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 21:51:38.759114 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 21:51:38.759114 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 06 21:51:38.759114 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:38.759114 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 510/1013] 10.222.0.18 () {56 vars in 1126 bytes} [Sat Aug 6 21:51:38 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:38.764614 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f3dbbb-1b4d-4ae9-8a4f-8acfdaf39563 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:38.814961 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:38.874907 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f3dbbb-1b4d-4ae9-8a4f-8acfdaf39563 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:38.909987 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06572a0f-b2ba-4522-878e-a27f1f0ce1e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:38.910592 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06572a0f-b2ba-4522-878e-a27f1f0ce1e1 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:38.910866 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06572a0f-b2ba-4522-878e-a27f1f0ce1e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:38.911131 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06572a0f-b2ba-4522-878e-a27f1f0ce1e1 None admin] PATH_INFO: `/v3/services` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:38.911758 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06572a0f-b2ba-4522-878e-a27f1f0ce1e1 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:38.912056 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06572a0f-b2ba-4522-878e-a27f1f0ce1e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:38.913449 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06572a0f-b2ba-4522-878e-a27f1f0ce1e1 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:38.918489 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 504/1014] 10.222.0.18 () {56 vars in 1122 bytes} [Sat Aug 6 21:51:38 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:51:38.924563 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44b51d4-ef10-464b-a279-f209d567dfb2 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:39.026897 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44b51d4-ef10-464b-a279-f209d567dfb2 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:39.059278 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea32dfb3-ef41-4f0c-9e1f-a69728c7b448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:39.059932 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea32dfb3-ef41-4f0c-9e1f-a69728c7b448 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:39.060153 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea32dfb3-ef41-4f0c-9e1f-a69728c7b448 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:39.060962 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea32dfb3-ef41-4f0c-9e1f-a69728c7b448 None admin] PATH_INFO: `/v3/services` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:39.061593 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea32dfb3-ef41-4f0c-9e1f-a69728c7b448 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:39.061885 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea32dfb3-ef41-4f0c-9e1f-a69728c7b448 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:39.063237 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea32dfb3-ef41-4f0c-9e1f-a69728c7b448 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:39.072810 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 511/1015] 10.222.0.18 () {56 vars in 1122 bytes} [Sat Aug 6 21:51:38 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:51:39.074069 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06572a0f-b2ba-4522-878e-a27f1f0ce1e1 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:39.212748 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06572a0f-b2ba-4522-878e-a27f1f0ce1e1 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:39.220384 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:39.249558 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0949a8bf-f83e-48f8-98fe-fb255586e627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:39.250157 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0949a8bf-f83e-48f8-98fe-fb255586e627 None admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:39.250432 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0949a8bf-f83e-48f8-98fe-fb255586e627 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:39.250700 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0949a8bf-f83e-48f8-98fe-fb255586e627 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:39.251358 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0949a8bf-f83e-48f8-98fe-fb255586e627 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:39.252740 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0949a8bf-f83e-48f8-98fe-fb255586e627 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:39.309383 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 505/1016] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 21:51:39 2022] POST /identity/v3/endpoints => generated 325 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 21:51:40.508739 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc3bbde-7320-4c66-ab19-799910d06383 None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:40.509289 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc3bbde-7320-4c66-ab19-799910d06383 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:40.509289 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc3bbde-7320-4c66-ab19-799910d06383 None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:40.510817 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 512/1017] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:51:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:51:40.516289 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d69e1c0-f769-4c66-a82a-04b090aa48b4 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:40.516705 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d69e1c0-f769-4c66-a82a-04b090aa48b4 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:40.516866 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d69e1c0-f769-4c66-a82a-04b090aa48b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:40.534843 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-2d69e1c0-f769-4c66-a82a-04b090aa48b4 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:51:40.536027 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-2d69e1c0-f769-4c66-a82a-04b090aa48b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:51:40.610098 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 506/1018] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:51:40 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 21:51:40.631664 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc3bbde-7320-4c66-ab19-799910d06383 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:40.641787 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:40.673079 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2fe80a-fc02-4e5e-8acf-f983c1a9b7c8 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:40.673386 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2fe80a-fc02-4e5e-8acf-f983c1a9b7c8 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:40.673612 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2fe80a-fc02-4e5e-8acf-f983c1a9b7c8 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:40.674946 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 507/1019] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:51:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 21:51:40.679846 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703fbb52-8b43-42c0-b7cb-ee51ad866e66 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:40.680134 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703fbb52-8b43-42c0-b7cb-ee51ad866e66 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:40.680422 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703fbb52-8b43-42c0-b7cb-ee51ad866e66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:40.699505 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-703fbb52-8b43-42c0-b7cb-ee51ad866e66 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:51:40.700718 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-703fbb52-8b43-42c0-b7cb-ee51ad866e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:51:40.742955 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc3bbde-7320-4c66-ab19-799910d06383 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:40.769601 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 508/1020] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:51:40 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 21:51:40.780248 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83725827-ae8e-461c-a61e-4df996b1920e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:40.781112 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83725827-ae8e-461c-a61e-4df996b1920e None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:40.781589 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83725827-ae8e-461c-a61e-4df996b1920e None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:40.782031 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83725827-ae8e-461c-a61e-4df996b1920e None admin] PATH_INFO: `/v3/services/image` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:40.782957 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83725827-ae8e-461c-a61e-4df996b1920e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:40.784578 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83725827-ae8e-461c-a61e-4df996b1920e None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:40.792938 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703fbb52-8b43-42c0-b7cb-ee51ad866e66 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:40.793628 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-83725827-ae8e-461c-a61e-4df996b1920e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 06 21:51:40.793628 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:40.793628 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 21:51:40.793628 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 21:51:40.793628 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 21:51:40.793628 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 21:51:40.793628 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 21:51:40.793628 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 21:51:40.793628 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 06 21:51:40.793628 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:40.793628 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 21:51:40.793628 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:40.793628 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:40.793628 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:40.793628 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:40.793628 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:40.793628 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:40.794670 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:40.794670 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:40.794670 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:40.794670 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:40.794670 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:40.794670 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:40.794670 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 21:51:40.794670 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 21:51:40.794670 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 21:51:40.794670 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 21:51:40.794670 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:40.794670 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:40.794670 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:40.794670 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:40.794670 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:40.794670 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:40.795623 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:40.795623 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:40.795623 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:40.795623 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:40.795623 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:40.795623 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:40.795623 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:40.795623 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:40.795623 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:40.795623 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:40.795623 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 21:51:40.795623 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 21:51:40.795623 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 06 21:51:40.795623 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:40.796389 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 513/1021] 10.222.0.18 () {56 vars in 1105 bytes} [Sat Aug 6 21:51:40 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:40.802128 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83725827-ae8e-461c-a61e-4df996b1920e None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:40.818811 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:40.877144 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:40.897865 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703fbb52-8b43-42c0-b7cb-ee51ad866e66 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:40.912169 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83725827-ae8e-461c-a61e-4df996b1920e None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:40.937204 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a8ba4b-8320-4e0a-84c7-75eb54420847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:40.938154 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a8ba4b-8320-4e0a-84c7-75eb54420847 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:40.938532 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a8ba4b-8320-4e0a-84c7-75eb54420847 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:40.938696 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a8ba4b-8320-4e0a-84c7-75eb54420847 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:40.947901 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c71b2d-c50e-4263-b58e-b23314f26a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:40.948703 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c71b2d-c50e-4263-b58e-b23314f26a42 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:40.948979 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a8ba4b-8320-4e0a-84c7-75eb54420847 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:51:40.949106 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c71b2d-c50e-4263-b58e-b23314f26a42 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:40.949274 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c71b2d-c50e-4263-b58e-b23314f26a42 None admin] PATH_INFO: `/v3/services` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:40.949888 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c71b2d-c50e-4263-b58e-b23314f26a42 None admin] RBAC: Adding query filter params (name=image) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:40.950281 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c71b2d-c50e-4263-b58e-b23314f26a42 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:40.952223 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c71b2d-c50e-4263-b58e-b23314f26a42 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:40.954796 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-a7a8ba4b-8320-4e0a-84c7-75eb54420847 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 06 21:51:40.954796 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:40.954796 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:40.954796 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:40.954796 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:40.954796 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:40.954796 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:40.954796 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:40.954796 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:40.954796 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:40.954796 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:40.954796 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:40.954796 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:51:40.954796 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:51:40.954796 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:51:40.954796 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:51:40.955734 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:51:40.955734 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:51:40.955734 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:51:40.955734 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:51:40.955734 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:40.955734 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:40.955734 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:40.955734 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:40.955734 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:40.955734 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:40.955734 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:40.955734 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:40.955734 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:40.955734 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:40.955734 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:40.955734 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:40.956639 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:40.956639 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:40.956639 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:40.956639 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:40.956639 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:51:40.956639 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:51:40.956639 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:51:40.956639 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:51:40.956639 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:51:40.956639 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:51:40.956639 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 06 21:51:40.956639 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:40.956639 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 509/1022] 10.222.0.18 () {56 vars in 1120 bytes} [Sat Aug 6 21:51:40 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 21:51:40.964477 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 514/1023] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:51:40 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:51:40.965165 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a8ba4b-8320-4e0a-84c7-75eb54420847 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:40.966296 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c71b2d-c50e-4263-b58e-b23314f26a42 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:41.058851 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:41.071726 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c71b2d-c50e-4263-b58e-b23314f26a42 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:41.080284 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a8ba4b-8320-4e0a-84c7-75eb54420847 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:41.107634 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6da71c-3791-4113-a141-8243e9eaf4dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:41.108532 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6da71c-3791-4113-a141-8243e9eaf4dd None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:41.108814 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6da71c-3791-4113-a141-8243e9eaf4dd None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:41.109087 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6da71c-3791-4113-a141-8243e9eaf4dd None admin] PATH_INFO: `/v3/services` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:41.109735 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6da71c-3791-4113-a141-8243e9eaf4dd None admin] RBAC: Adding query filter params (type=image) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:41.110109 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6da71c-3791-4113-a141-8243e9eaf4dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:41.111745 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6da71c-3791-4113-a141-8243e9eaf4dd None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:41.113091 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:41.121833 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 515/1024] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:51:40 2022] GET /identity/v3/services?type=image => generated 350 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:41.123787 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6da71c-3791-4113-a141-8243e9eaf4dd None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:41.126257 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36ffb8c-c7cc-4f13-9219-c25d9b5f2691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:41.126788 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36ffb8c-c7cc-4f13-9219-c25d9b5f2691 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:41.127829 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36ffb8c-c7cc-4f13-9219-c25d9b5f2691 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:41.127829 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36ffb8c-c7cc-4f13-9219-c25d9b5f2691 None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:41.128024 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36ffb8c-c7cc-4f13-9219-c25d9b5f2691 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:41.128283 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36ffb8c-c7cc-4f13-9219-c25d9b5f2691 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:41.129823 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36ffb8c-c7cc-4f13-9219-c25d9b5f2691 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:41.143510 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 510/1025] 10.222.0.18 () {56 vars in 1115 bytes} [Sat Aug 6 21:51:40 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:51:41.148959 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36ffb8c-c7cc-4f13-9219-c25d9b5f2691 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:41.236152 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6da71c-3791-4113-a141-8243e9eaf4dd None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:41.255171 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36ffb8c-c7cc-4f13-9219-c25d9b5f2691 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:41.273378 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bb6d7a-154d-449d-85cc-a0cc6be0778c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:41.273986 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bb6d7a-154d-449d-85cc-a0cc6be0778c None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:41.274287 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bb6d7a-154d-449d-85cc-a0cc6be0778c None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:41.274576 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bb6d7a-154d-449d-85cc-a0cc6be0778c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:41.275273 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bb6d7a-154d-449d-85cc-a0cc6be0778c None admin] RBAC: Adding query filter params (interface=public, service_id=c35bcb52c6144adb9a4397d9287d3339, region_id=RegionOne) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:41.275575 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bb6d7a-154d-449d-85cc-a0cc6be0778c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:41.277003 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bb6d7a-154d-449d-85cc-a0cc6be0778c None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:41.283117 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 516/1026] 10.222.0.18 () {56 vars in 1251 bytes} [Sat Aug 6 21:51:41 2022] GET /identity/v3/endpoints?service_id=c35bcb52c6144adb9a4397d9287d3339&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:51:41.288377 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bb6d7a-154d-449d-85cc-a0cc6be0778c None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:41.297779 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eea0b16-cb4e-49b7-8ebc-60662180afa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:41.298416 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eea0b16-cb4e-49b7-8ebc-60662180afa6 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:41.298694 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eea0b16-cb4e-49b7-8ebc-60662180afa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:41.298962 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eea0b16-cb4e-49b7-8ebc-60662180afa6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:41.310181 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eea0b16-cb4e-49b7-8ebc-60662180afa6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:41.311621 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eea0b16-cb4e-49b7-8ebc-60662180afa6 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:41.314645 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:41.327517 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-4eea0b16-cb4e-49b7-8ebc-60662180afa6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:51:41.327517 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:41.327517 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:41.327517 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:41.327517 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:41.327517 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:41.327517 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:41.327517 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:41.327517 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:41.327517 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:41.327517 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:41.327517 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:41.327517 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:51:41.327517 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:51:41.327517 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:51:41.327517 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:51:41.328826 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:41.328826 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:41.328826 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:41.328826 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:41.328826 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:41.328826 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:41.328826 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:41.328826 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:41.328826 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:41.328826 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:41.328826 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:41.328826 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:41.328826 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:41.328826 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:41.328826 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:41.328826 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:41.328826 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:51:41.329826 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:51:41.329826 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:51:41.329826 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:41.329826 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 511/1027] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:51:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:41.334174 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eea0b16-cb4e-49b7-8ebc-60662180afa6 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:41.362642 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:41.403571 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bb6d7a-154d-449d-85cc-a0cc6be0778c None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:41.435839 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eea0b16-cb4e-49b7-8ebc-60662180afa6 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:41.437942 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c634ac2-fab7-479b-9987-a2f254da3714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:41.438544 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c634ac2-fab7-479b-9987-a2f254da3714 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:41.438819 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c634ac2-fab7-479b-9987-a2f254da3714 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:41.439090 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c634ac2-fab7-479b-9987-a2f254da3714 None admin] PATH_INFO: `/v3/services` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:41.439688 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c634ac2-fab7-479b-9987-a2f254da3714 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:41.441074 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c634ac2-fab7-479b-9987-a2f254da3714 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:41.447460 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 517/1028] 10.222.0.18 () {56 vars in 1087 bytes} [Sat Aug 6 21:51:41 2022] GET /identity/v3/services => generated 1957 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 21:51:41.448684 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0aaf52-e977-4eee-bb4f-ac80f955816d None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:41.448969 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0aaf52-e977-4eee-bb4f-ac80f955816d None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:41.449245 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0aaf52-e977-4eee-bb4f-ac80f955816d None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:41.450454 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 518/1029] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:51:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:51:41.456252 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae24da6b-fe8c-46cc-a433-66fbdf388728 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:41.456571 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae24da6b-fe8c-46cc-a433-66fbdf388728 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:41.456848 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae24da6b-fe8c-46cc-a433-66fbdf388728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:41.477005 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-ae24da6b-fe8c-46cc-a433-66fbdf388728 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:51:41.478261 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-ae24da6b-fe8c-46cc-a433-66fbdf388728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:51:41.480717 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d448df0-eeb4-43dc-ae24-3f155b714271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:41.482980 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d448df0-eeb4-43dc-ae24-3f155b714271 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:41.482980 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d448df0-eeb4-43dc-ae24-3f155b714271 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:41.482980 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d448df0-eeb4-43dc-ae24-3f155b714271 None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:41.483304 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d448df0-eeb4-43dc-ae24-3f155b714271 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:41.483605 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d448df0-eeb4-43dc-ae24-3f155b714271 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:41.485239 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d448df0-eeb4-43dc-ae24-3f155b714271 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:41.499154 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 512/1030] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:51:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:51:41.505102 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d448df0-eeb4-43dc-ae24-3f155b714271 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:41.561254 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 519/1031] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:51:41 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 21:51:41.578135 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:41.586193 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae24da6b-fe8c-46cc-a433-66fbdf388728 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:41.627277 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d448df0-eeb4-43dc-ae24-3f155b714271 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:41.651720 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:41.663906 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db91a7b-70a6-4b6a-9baa-ca0109825d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:41.663906 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db91a7b-70a6-4b6a-9baa-ca0109825d8f None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:41.663906 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db91a7b-70a6-4b6a-9baa-ca0109825d8f None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:41.663906 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db91a7b-70a6-4b6a-9baa-ca0109825d8f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:41.672135 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db91a7b-70a6-4b6a-9baa-ca0109825d8f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:41.673639 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db91a7b-70a6-4b6a-9baa-ca0109825d8f None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:41.685850 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-7db91a7b-70a6-4b6a-9baa-ca0109825d8f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 06 21:51:41.685850 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:41.685850 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:41.685850 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:41.685850 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:41.685850 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:41.685850 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:41.685850 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:41.685850 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:41.685850 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:41.685850 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:41.685850 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:41.685850 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 21:51:41.685850 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 21:51:41.685850 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 21:51:41.685850 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 21:51:41.687011 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:41.687011 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:41.687011 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 21:51:41.687011 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 21:51:41.687011 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 21:51:41.687011 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 21:51:41.687011 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:41.687011 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:41.687011 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:41.687011 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:41.687011 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:41.687011 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:41.687011 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:41.687011 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:41.687011 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:41.687011 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:41.687011 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:41.687011 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:41.688171 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:41.688171 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:41.688171 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 21:51:41.688171 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 21:51:41.688171 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 21:51:41.688171 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 21:51:41.688171 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 21:51:41.688171 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 21:51:41.688171 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 06 21:51:41.688171 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:41.691735 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 513/1032] 10.222.0.18 () {56 vars in 1093 bytes} [Sat Aug 6 21:51:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:41.692608 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db91a7b-70a6-4b6a-9baa-ca0109825d8f None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:41.707078 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae24da6b-fe8c-46cc-a433-66fbdf388728 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:41.742874 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a150dc65-542d-452c-ae7e-d9d3d7d80342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:41.743499 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a150dc65-542d-452c-ae7e-d9d3d7d80342 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:41.743776 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a150dc65-542d-452c-ae7e-d9d3d7d80342 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:41.744048 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a150dc65-542d-452c-ae7e-d9d3d7d80342 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:41.745306 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a150dc65-542d-452c-ae7e-d9d3d7d80342 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:41.747035 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a150dc65-542d-452c-ae7e-d9d3d7d80342 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:41.749763 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 520/1033] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:51:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:51:41.755229 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a150dc65-542d-452c-ae7e-d9d3d7d80342 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:41.808737 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db91a7b-70a6-4b6a-9baa-ca0109825d8f None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:41.830016 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:41.851008 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0b394e-460b-4f06-b532-3e8dc89664bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:41.851781 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0b394e-460b-4f06-b532-3e8dc89664bb None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:41.852307 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0b394e-460b-4f06-b532-3e8dc89664bb None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:41.853237 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0b394e-460b-4f06-b532-3e8dc89664bb None admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:41.854121 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0b394e-460b-4f06-b532-3e8dc89664bb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:41.854435 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0b394e-460b-4f06-b532-3e8dc89664bb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:41.855525 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a150dc65-542d-452c-ae7e-d9d3d7d80342 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:41.855881 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0b394e-460b-4f06-b532-3e8dc89664bb None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:41.875533 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 514/1034] 10.222.0.18 () {56 vars in 1133 bytes} [Sat Aug 6 21:51:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:41.881125 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0b394e-460b-4f06-b532-3e8dc89664bb None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:41.884462 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:41.896714 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242d83c0-f49f-4ce6-91de-fc884db2b022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:41.897321 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d83c0-f49f-4ce6-91de-fc884db2b022 None admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:41.897601 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d83c0-f49f-4ce6-91de-fc884db2b022 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:41.897870 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d83c0-f49f-4ce6-91de-fc884db2b022 None admin] PATH_INFO: `/v3/projects` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:41.898512 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242d83c0-f49f-4ce6-91de-fc884db2b022 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:41.899923 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242d83c0-f49f-4ce6-91de-fc884db2b022 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:41.988727 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0b394e-460b-4f06-b532-3e8dc89664bb None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:41.997318 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 521/1035] 10.222.0.18 () {60 vars in 1141 bytes} [Sat Aug 6 21:51:41 2022] POST /identity/v3/projects => generated 311 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 21:51:42.025039 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5788b88b-dcae-4e8a-928c-316534e44558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:42.025666 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5788b88b-dcae-4e8a-928c-316534e44558 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:42.025939 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5788b88b-dcae-4e8a-928c-316534e44558 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:42.026206 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5788b88b-dcae-4e8a-928c-316534e44558 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:42.073214 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:42.100526 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5788b88b-dcae-4e8a-928c-316534e44558 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:42.101999 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5788b88b-dcae-4e8a-928c-316534e44558 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:42.211705 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 515/1036] 10.222.0.18 () {58 vars in 1282 bytes} [Sat Aug 6 21:51:41 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 06 21:51:42.217884 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242d83c0-f49f-4ce6-91de-fc884db2b022 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:42.320691 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242d83c0-f49f-4ce6-91de-fc884db2b022 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:42.361433 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c275cf4-6e7c-4f44-99d2-e02a9f541afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:42.362078 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c275cf4-6e7c-4f44-99d2-e02a9f541afa None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:42.362357 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c275cf4-6e7c-4f44-99d2-e02a9f541afa None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:42.362630 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c275cf4-6e7c-4f44-99d2-e02a9f541afa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:42.374528 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c275cf4-6e7c-4f44-99d2-e02a9f541afa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:42.376228 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c275cf4-6e7c-4f44-99d2-e02a9f541afa None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:42.391369 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-4c275cf4-6e7c-4f44-99d2-e02a9f541afa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:51:42.391369 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:42.391369 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:42.391369 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:42.391369 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:42.391369 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:42.391369 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:42.391369 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:42.391369 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:42.391369 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:42.391369 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:42.391369 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:42.391369 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:51:42.391369 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:51:42.391369 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:51:42.391369 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:51:42.392313 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:42.392313 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:42.392313 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:42.392313 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:42.392313 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:42.392313 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:42.392313 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:42.392313 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:42.392313 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:42.392313 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:42.392313 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:42.392313 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:42.392313 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:42.392313 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:42.392313 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:42.392313 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:42.392313 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:51:42.393311 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:51:42.393311 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:51:42.393311 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:42.398377 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 522/1037] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:51:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:42.400030 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5788b88b-dcae-4e8a-928c-316534e44558 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:42.476473 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:42.503038 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5788b88b-dcae-4e8a-928c-316534e44558 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:42.539365 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eec1eb0-c385-4e85-a3b1-180e66a3e862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:42.540269 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eec1eb0-c385-4e85-a3b1-180e66a3e862 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:42.540761 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eec1eb0-c385-4e85-a3b1-180e66a3e862 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:42.541186 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eec1eb0-c385-4e85-a3b1-180e66a3e862 None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:42.542028 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eec1eb0-c385-4e85-a3b1-180e66a3e862 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:42.542480 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eec1eb0-c385-4e85-a3b1-180e66a3e862 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:42.544018 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eec1eb0-c385-4e85-a3b1-180e66a3e862 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:42.562255 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 516/1038] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:51:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:42.564541 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c275cf4-6e7c-4f44-99d2-e02a9f541afa None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:42.595398 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:42.670504 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c275cf4-6e7c-4f44-99d2-e02a9f541afa None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:42.706462 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158333f8-b4f1-4e94-96fa-f53040f49914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:42.707342 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158333f8-b4f1-4e94-96fa-f53040f49914 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:42.707826 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158333f8-b4f1-4e94-96fa-f53040f49914 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:42.708246 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158333f8-b4f1-4e94-96fa-f53040f49914 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:42.715081 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158333f8-b4f1-4e94-96fa-f53040f49914 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:42.717464 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158333f8-b4f1-4e94-96fa-f53040f49914 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:42.728812 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-158333f8-b4f1-4e94-96fa-f53040f49914 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 21:51:42.728812 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:42.728812 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:42.728812 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:42.728812 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:42.728812 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:42.728812 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:42.728812 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:42.728812 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:42.728812 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:42.728812 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:42.728812 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:42.728812 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 21:51:42.728812 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 21:51:42.728812 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 21:51:42.729872 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 21:51:42.729872 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:42.729872 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:42.729872 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:42.729872 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:42.729872 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:42.729872 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:42.729872 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:42.729872 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:42.729872 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:42.729872 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:42.729872 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:42.729872 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:42.729872 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:42.729872 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:42.729872 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:42.729872 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:42.731179 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 21:51:42.731179 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 21:51:42.731179 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 21:51:42.731179 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 21:51:42.731179 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 21:51:42.731179 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 21:51:42.731179 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 21:51:42.731179 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:42.731179 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 523/1039] 10.222.0.18 () {56 vars in 1111 bytes} [Sat Aug 6 21:51:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:42.736951 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eec1eb0-c385-4e85-a3b1-180e66a3e862 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:42.839515 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eec1eb0-c385-4e85-a3b1-180e66a3e862 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:42.873903 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8b01ec-dd5d-4c54-afc0-a6e67629ba22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:42.874493 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8b01ec-dd5d-4c54-afc0-a6e67629ba22 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:42.874774 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8b01ec-dd5d-4c54-afc0-a6e67629ba22 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:42.875068 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8b01ec-dd5d-4c54-afc0-a6e67629ba22 None admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:42.875800 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8b01ec-dd5d-4c54-afc0-a6e67629ba22 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:42.876106 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8b01ec-dd5d-4c54-afc0-a6e67629ba22 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:42.877572 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8b01ec-dd5d-4c54-afc0-a6e67629ba22 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:42.879174 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:42.890768 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 517/1040] 10.222.0.18 () {56 vars in 1148 bytes} [Sat Aug 6 21:51:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:42.896746 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158333f8-b4f1-4e94-96fa-f53040f49914 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:42.991306 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:43.004126 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158333f8-b4f1-4e94-96fa-f53040f49914 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:43.040495 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4648bfc-661e-4e10-9d7c-f785337f8749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:43.041247 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4648bfc-661e-4e10-9d7c-f785337f8749 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:43.041629 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4648bfc-661e-4e10-9d7c-f785337f8749 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:43.042013 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4648bfc-661e-4e10-9d7c-f785337f8749 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:43.042836 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4648bfc-661e-4e10-9d7c-f785337f8749 None admin] RBAC: Adding query filter params (role.id=08bf738879a941dab8ac6323043dc991, scope.project.id=4bb6307eb0644d85a1d1daaa61352697, user.id=2a1319d33e9144859c6010843be312a3) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:43.043265 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4648bfc-661e-4e10-9d7c-f785337f8749 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:43.045454 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4648bfc-661e-4e10-9d7c-f785337f8749 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:43.056483 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 524/1041] 10.222.0.18 () {56 vars in 1374 bytes} [Sat Aug 6 21:51:42 2022] GET /identity/v3/role_assignments?user.id=2a1319d33e9144859c6010843be312a3&scope.project.id=4bb6307eb0644d85a1d1daaa61352697&role.id=08bf738879a941dab8ac6323043dc991 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:43.454926 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f07d91-f80b-4b71-88da-edc57b85e350 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:43.455254 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f07d91-f80b-4b71-88da-edc57b85e350 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:43.455485 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f07d91-f80b-4b71-88da-edc57b85e350 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:43.456950 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 518/1042] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:51:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:51:43.462020 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f40a4c-9987-4faa-b5fb-6ef74a1fdebe None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:43.462301 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f40a4c-9987-4faa-b5fb-6ef74a1fdebe None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:43.462568 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f40a4c-9987-4faa-b5fb-6ef74a1fdebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:43.480697 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-60f40a4c-9987-4faa-b5fb-6ef74a1fdebe None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:51:43.481942 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-60f40a4c-9987-4faa-b5fb-6ef74a1fdebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:51:43.554814 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 525/1043] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:51:43 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 21:51:43.578083 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f07d91-f80b-4b71-88da-edc57b85e350 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:43.682082 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f07d91-f80b-4b71-88da-edc57b85e350 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:43.717875 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cf681f-3df3-4d02-a932-d4c508d09607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:43.718506 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf681f-3df3-4d02-a932-d4c508d09607 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:43.718786 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf681f-3df3-4d02-a932-d4c508d09607 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:43.719060 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf681f-3df3-4d02-a932-d4c508d09607 None admin] PATH_INFO: `/v3/services/image` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:43.719640 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cf681f-3df3-4d02-a932-d4c508d09607 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:43.721099 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cf681f-3df3-4d02-a932-d4c508d09607 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:43.729893 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-d7cf681f-3df3-4d02-a932-d4c508d09607 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 06 21:51:43.729893 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:43.729893 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 21:51:43.729893 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 21:51:43.729893 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 21:51:43.729893 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 21:51:43.729893 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 21:51:43.729893 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 21:51:43.729893 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 06 21:51:43.729893 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:43.729893 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 21:51:43.729893 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:43.729893 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:43.729893 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:43.729893 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:43.729893 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:43.729893 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:43.730883 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:43.730883 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:43.730883 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:43.730883 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:43.730883 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:43.730883 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:43.730883 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 21:51:43.730883 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 21:51:43.730883 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 21:51:43.730883 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 21:51:43.730883 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:43.730883 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:43.730883 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:43.730883 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:43.730883 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:43.730883 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:43.731746 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:43.731746 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:43.731746 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:43.731746 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:43.731746 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:43.731746 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:43.731746 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:43.731746 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:43.731746 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:43.731746 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:43.731746 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 21:51:43.731746 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 21:51:43.731746 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 06 21:51:43.731746 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:43.734125 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 519/1044] 10.222.0.18 () {56 vars in 1105 bytes} [Sat Aug 6 21:51:43 2022] GET /identity/v3/services/image => generated 86 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:43.737994 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f40a4c-9987-4faa-b5fb-6ef74a1fdebe None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:43.820752 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:43.839255 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f40a4c-9987-4faa-b5fb-6ef74a1fdebe None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:43.874545 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980c8075-371b-4f5d-bcb7-81d80f99c70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:43.875149 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980c8075-371b-4f5d-bcb7-81d80f99c70e None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:43.875435 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980c8075-371b-4f5d-bcb7-81d80f99c70e None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:43.875707 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980c8075-371b-4f5d-bcb7-81d80f99c70e None admin] PATH_INFO: `/v3/services` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:43.876339 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980c8075-371b-4f5d-bcb7-81d80f99c70e None admin] RBAC: Adding query filter params (name=image) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:43.876670 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980c8075-371b-4f5d-bcb7-81d80f99c70e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:43.878032 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980c8075-371b-4f5d-bcb7-81d80f99c70e None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:43.887567 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 526/1045] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:51:43 2022] GET /identity/v3/services?name=image => generated 122 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:51:43.889189 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cf681f-3df3-4d02-a932-d4c508d09607 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:43.949733 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:43.993734 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cf681f-3df3-4d02-a932-d4c508d09607 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:44.027776 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffd49f3-e061-4fcf-b1c8-fb8cf5e1af81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:44.028390 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffd49f3-e061-4fcf-b1c8-fb8cf5e1af81 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:44.028719 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffd49f3-e061-4fcf-b1c8-fb8cf5e1af81 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:44.028995 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffd49f3-e061-4fcf-b1c8-fb8cf5e1af81 None admin] PATH_INFO: `/v3/services` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:44.029797 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffd49f3-e061-4fcf-b1c8-fb8cf5e1af81 None admin] RBAC: Adding query filter params (type=image) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:44.030102 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffd49f3-e061-4fcf-b1c8-fb8cf5e1af81 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:44.031465 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffd49f3-e061-4fcf-b1c8-fb8cf5e1af81 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:44.040549 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 520/1046] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:51:43 2022] GET /identity/v3/services?type=image => generated 350 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:51:44.042664 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980c8075-371b-4f5d-bcb7-81d80f99c70e None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:44.059508 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039bf14c-c296-4f8e-ba4d-4d3e58ab2ce4 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:44.059787 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039bf14c-c296-4f8e-ba4d-4d3e58ab2ce4 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:44.060627 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039bf14c-c296-4f8e-ba4d-4d3e58ab2ce4 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:44.062099 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 521/1047] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:51:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 21:51:44.066646 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3abc831-09ec-45b5-a9d6-0e5aae47ff2e None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:44.066929 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3abc831-09ec-45b5-a9d6-0e5aae47ff2e None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:44.067198 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3abc831-09ec-45b5-a9d6-0e5aae47ff2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:44.084723 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-b3abc831-09ec-45b5-a9d6-0e5aae47ff2e None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:51:44.085886 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-b3abc831-09ec-45b5-a9d6-0e5aae47ff2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:51:44.163897 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 522/1048] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:51:44 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 21:51:44.166444 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980c8075-371b-4f5d-bcb7-81d80f99c70e None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:44.188193 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3abc831-09ec-45b5-a9d6-0e5aae47ff2e None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:44.201677 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63afcb56-3c42-4498-8e49-742e8e5b4a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:44.202213 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63afcb56-3c42-4498-8e49-742e8e5b4a9d None admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:44.202483 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63afcb56-3c42-4498-8e49-742e8e5b4a9d None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:44.202750 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63afcb56-3c42-4498-8e49-742e8e5b4a9d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:44.203400 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63afcb56-3c42-4498-8e49-742e8e5b4a9d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:44.204799 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63afcb56-3c42-4498-8e49-742e8e5b4a9d None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:44.242596 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:44.257847 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 527/1049] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 21:51:44 2022] POST /identity/v3/endpoints => generated 327 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 21:51:44.299137 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3abc831-09ec-45b5-a9d6-0e5aae47ff2e None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:44.345047 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcadf69-96f9-44e6-976f-2240f94e679f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:44.345693 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcadf69-96f9-44e6-976f-2240f94e679f None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:44.349000 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcadf69-96f9-44e6-976f-2240f94e679f None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:44.349000 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcadf69-96f9-44e6-976f-2240f94e679f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:44.349681 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcadf69-96f9-44e6-976f-2240f94e679f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:44.351641 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcadf69-96f9-44e6-976f-2240f94e679f None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:44.353227 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 523/1050] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:51:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:44.361455 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63afcb56-3c42-4498-8e49-742e8e5b4a9d None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:44.364458 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:44.471665 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63afcb56-3c42-4498-8e49-742e8e5b4a9d None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:44.506361 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27e53c7-9aba-4fad-8335-5c8b5797348b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:44.506965 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27e53c7-9aba-4fad-8335-5c8b5797348b None admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:44.507241 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27e53c7-9aba-4fad-8335-5c8b5797348b None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:44.507510 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27e53c7-9aba-4fad-8335-5c8b5797348b None admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:44.508160 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27e53c7-9aba-4fad-8335-5c8b5797348b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:44.509608 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27e53c7-9aba-4fad-8335-5c8b5797348b None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:44.588889 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 528/1051] 10.222.0.18 () {60 vars in 1132 bytes} [Sat Aug 6 21:51:44 2022] POST /identity/v3/users => generated 285 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 21:51:45.067906 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a87e8-4a43-4a46-a20d-80e02d3a597e None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:45.068645 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a87e8-4a43-4a46-a20d-80e02d3a597e None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:45.068645 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a87e8-4a43-4a46-a20d-80e02d3a597e None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:45.069848 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 524/1052] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:51:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:51:45.074883 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2809c3bf-1085-4ff6-b245-89d9551ca03a None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:45.075280 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2809c3bf-1085-4ff6-b245-89d9551ca03a None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:45.075459 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2809c3bf-1085-4ff6-b245-89d9551ca03a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:45.094072 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-2809c3bf-1085-4ff6-b245-89d9551ca03a None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:51:45.095263 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-2809c3bf-1085-4ff6-b245-89d9551ca03a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:51:45.151027 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:45.165809 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 529/1053] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:51:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 21:51:45.188814 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7a87e8-4a43-4a46-a20d-80e02d3a597e None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:45.291818 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7a87e8-4a43-4a46-a20d-80e02d3a597e None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:45.308846 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:45.328225 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de76f617-85bd-478f-8e0f-92d82c33e90c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:45.328817 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de76f617-85bd-478f-8e0f-92d82c33e90c None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:45.329110 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de76f617-85bd-478f-8e0f-92d82c33e90c None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:45.329377 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de76f617-85bd-478f-8e0f-92d82c33e90c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:45.340415 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de76f617-85bd-478f-8e0f-92d82c33e90c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:51:45.345115 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-de76f617-85bd-478f-8e0f-92d82c33e90c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 06 21:51:45.345115 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:45.345115 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:45.345115 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:45.345115 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:45.345115 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:45.345115 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:45.345115 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:45.345115 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:45.345115 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:45.345115 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:45.345115 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:45.345115 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:51:45.345115 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:51:45.345115 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:51:45.345115 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:51:45.346074 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:51:45.346074 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:51:45.346074 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:51:45.346074 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:51:45.346074 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:45.346074 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:45.346074 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:45.346074 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:45.346074 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:45.346074 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:45.346074 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:45.346074 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:45.346074 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:45.346074 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:45.346074 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:45.346074 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:45.346955 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:45.346955 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:45.346955 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:45.346955 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:45.346955 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:51:45.346955 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:51:45.346955 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:51:45.346955 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:51:45.346955 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:51:45.346955 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:51:45.346955 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 06 21:51:45.346955 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:45.351860 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 525/1054] 10.222.0.18 () {56 vars in 1120 bytes} [Sat Aug 6 21:51:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:45.352775 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2809c3bf-1085-4ff6-b245-89d9551ca03a None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:45.460792 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2809c3bf-1085-4ff6-b245-89d9551ca03a None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:45.496572 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b562a51f-9c22-4ff7-9d01-a6605e98fac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:45.497521 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b562a51f-9c22-4ff7-9d01-a6605e98fac3 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:45.498204 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b562a51f-9c22-4ff7-9d01-a6605e98fac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:45.498674 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b562a51f-9c22-4ff7-9d01-a6605e98fac3 None admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:45.499521 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b562a51f-9c22-4ff7-9d01-a6605e98fac3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:45.500013 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b562a51f-9c22-4ff7-9d01-a6605e98fac3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:45.501670 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b562a51f-9c22-4ff7-9d01-a6605e98fac3 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:45.515055 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 530/1055] 10.222.0.18 () {56 vars in 1115 bytes} [Sat Aug 6 21:51:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:45.516657 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de76f617-85bd-478f-8e0f-92d82c33e90c None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:45.622100 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de76f617-85bd-478f-8e0f-92d82c33e90c None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:45.657125 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ebe1cf-e402-44a6-b027-b0d2389d5ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:45.657999 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ebe1cf-e402-44a6-b027-b0d2389d5ba3 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:45.658482 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ebe1cf-e402-44a6-b027-b0d2389d5ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:45.658934 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ebe1cf-e402-44a6-b027-b0d2389d5ba3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:45.669080 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ebe1cf-e402-44a6-b027-b0d2389d5ba3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:45.670973 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ebe1cf-e402-44a6-b027-b0d2389d5ba3 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:45.686315 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-17ebe1cf-e402-44a6-b027-b0d2389d5ba3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:51:45.686315 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:45.686315 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:45.686315 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:45.686315 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:45.686315 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:45.686315 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:45.686315 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:45.686315 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:45.686315 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:45.686315 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:45.686315 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:45.686315 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:51:45.686315 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:51:45.686315 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:51:45.686315 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:51:45.687276 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:45.687276 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:45.687276 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:45.687276 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:45.687276 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:45.687276 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:45.687276 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:45.687276 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:45.687276 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:45.687276 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:45.687276 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:45.687276 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:45.687276 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:45.687276 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:45.687276 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:45.687276 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:45.687276 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:51:45.688211 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:51:45.688211 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:51:45.688211 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:45.694132 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 526/1056] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:51:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:45.695274 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b562a51f-9c22-4ff7-9d01-a6605e98fac3 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:45.750365 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:45.801052 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b562a51f-9c22-4ff7-9d01-a6605e98fac3 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:45.834801 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf889b3-fb37-4a32-a5e5-272928b3f544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:45.835405 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf889b3-fb37-4a32-a5e5-272928b3f544 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:45.835683 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf889b3-fb37-4a32-a5e5-272928b3f544 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:45.835955 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf889b3-fb37-4a32-a5e5-272928b3f544 None admin] PATH_INFO: `/v3/domains` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:45.836622 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf889b3-fb37-4a32-a5e5-272928b3f544 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:45.836924 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf889b3-fb37-4a32-a5e5-272928b3f544 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:45.838275 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf889b3-fb37-4a32-a5e5-272928b3f544 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:45.855119 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 531/1057] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:51:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:45.856247 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ebe1cf-e402-44a6-b027-b0d2389d5ba3 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:45.886564 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:45.959440 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ebe1cf-e402-44a6-b027-b0d2389d5ba3 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:45.993163 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c11c95-cb16-4b10-b3cd-6e3aaad8505a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:45.993791 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c11c95-cb16-4b10-b3cd-6e3aaad8505a None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:45.994069 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c11c95-cb16-4b10-b3cd-6e3aaad8505a None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:45.994338 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c11c95-cb16-4b10-b3cd-6e3aaad8505a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:46.002352 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c11c95-cb16-4b10-b3cd-6e3aaad8505a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:46.003795 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c11c95-cb16-4b10-b3cd-6e3aaad8505a None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:46.015418 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-f6c11c95-cb16-4b10-b3cd-6e3aaad8505a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 06 21:51:46.015418 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:46.015418 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:46.015418 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:46.015418 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:46.015418 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:46.015418 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:46.015418 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:46.015418 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:46.015418 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:46.015418 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:46.015418 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:46.015418 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 21:51:46.015418 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 21:51:46.015418 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 21:51:46.015418 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 21:51:46.016486 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:46.016486 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:46.016486 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 21:51:46.016486 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 21:51:46.016486 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 21:51:46.016486 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 21:51:46.016486 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:46.016486 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:46.016486 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:46.016486 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:46.016486 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:46.016486 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:46.016486 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:46.016486 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:46.016486 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:46.016486 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:46.016486 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:46.016486 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:46.017478 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:46.017478 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:46.017478 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 21:51:46.017478 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 21:51:46.017478 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 21:51:46.017478 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 21:51:46.017478 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 21:51:46.017478 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 21:51:46.017478 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 06 21:51:46.017478 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:46.017478 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 527/1058] 10.222.0.18 () {56 vars in 1093 bytes} [Sat Aug 6 21:51:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:46.024007 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf889b3-fb37-4a32-a5e5-272928b3f544 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:46.126249 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf889b3-fb37-4a32-a5e5-272928b3f544 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:46.137142 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:46.167107 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6456dc69-bfa3-4dc7-b0ff-3b2beb1245ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:46.167726 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6456dc69-bfa3-4dc7-b0ff-3b2beb1245ca None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:46.168001 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6456dc69-bfa3-4dc7-b0ff-3b2beb1245ca None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:46.168270 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6456dc69-bfa3-4dc7-b0ff-3b2beb1245ca None admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:46.169037 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6456dc69-bfa3-4dc7-b0ff-3b2beb1245ca None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:46.169910 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6456dc69-bfa3-4dc7-b0ff-3b2beb1245ca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:46.171404 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6456dc69-bfa3-4dc7-b0ff-3b2beb1245ca None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:46.194141 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 532/1059] 10.222.0.18 () {56 vars in 1133 bytes} [Sat Aug 6 21:51:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:46.195418 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c11c95-cb16-4b10-b3cd-6e3aaad8505a None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:46.290190 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:46.303557 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c11c95-cb16-4b10-b3cd-6e3aaad8505a None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:46.319798 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b397f26-7a47-423e-ab28-27ca4f48aef2 None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:46.320178 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b397f26-7a47-423e-ab28-27ca4f48aef2 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:46.320569 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b397f26-7a47-423e-ab28-27ca4f48aef2 None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:46.322041 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 533/1060] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:51:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 21:51:46.327702 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35042dbe-6b8e-4cc4-b81c-f7b17e9bc9ca None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:46.328105 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35042dbe-6b8e-4cc4-b81c-f7b17e9bc9ca None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:46.328383 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35042dbe-6b8e-4cc4-b81c-f7b17e9bc9ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:46.340549 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb81e82-27f3-4646-9c73-8c175cd3c6cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:46.342414 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb81e82-27f3-4646-9c73-8c175cd3c6cf None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:46.342869 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb81e82-27f3-4646-9c73-8c175cd3c6cf None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:46.343321 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb81e82-27f3-4646-9c73-8c175cd3c6cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:46.347254 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-35042dbe-6b8e-4cc4-b81c-f7b17e9bc9ca None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:51:46.348848 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-35042dbe-6b8e-4cc4-b81c-f7b17e9bc9ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:51:46.357607 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb81e82-27f3-4646-9c73-8c175cd3c6cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:46.359514 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb81e82-27f3-4646-9c73-8c175cd3c6cf None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:46.374574 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-7fb81e82-27f3-4646-9c73-8c175cd3c6cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:51:46.374574 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:46.374574 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:46.374574 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:46.374574 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:46.374574 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:46.374574 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:46.374574 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:46.374574 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:46.374574 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:46.374574 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:46.374574 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:46.374574 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:51:46.374574 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:51:46.374574 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:51:46.374574 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:51:46.375744 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:46.375744 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:46.375744 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:46.375744 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:46.375744 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:46.375744 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:46.375744 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:46.375744 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:46.375744 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:46.375744 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:46.375744 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:46.375744 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:46.375744 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:46.375744 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:46.375744 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:46.375744 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:46.375744 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:51:46.377001 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:51:46.377001 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:51:46.377001 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:46.380447 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 528/1061] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:51:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:46.381299 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb81e82-27f3-4646-9c73-8c175cd3c6cf None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:46.428522 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 534/1062] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:51:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 21:51:46.448914 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35042dbe-6b8e-4cc4-b81c-f7b17e9bc9ca None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:46.492265 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb81e82-27f3-4646-9c73-8c175cd3c6cf None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:46.528521 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0f6fd7-eabd-4171-b6df-3c79970a555b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:46.529804 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0f6fd7-eabd-4171-b6df-3c79970a555b None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:46.529919 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0f6fd7-eabd-4171-b6df-3c79970a555b None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:46.530168 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0f6fd7-eabd-4171-b6df-3c79970a555b None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:46.530803 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0f6fd7-eabd-4171-b6df-3c79970a555b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:46.531103 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0f6fd7-eabd-4171-b6df-3c79970a555b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:46.532505 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0f6fd7-eabd-4171-b6df-3c79970a555b None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:46.541391 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:46.545433 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 529/1063] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:51:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:46.551103 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0f6fd7-eabd-4171-b6df-3c79970a555b None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:46.557084 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:46.561407 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35042dbe-6b8e-4cc4-b81c-f7b17e9bc9ca None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:46.598844 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f344a2f-4015-4626-a5de-2dc8421f10e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:46.599456 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f344a2f-4015-4626-a5de-2dc8421f10e0 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:46.599732 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f344a2f-4015-4626-a5de-2dc8421f10e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:46.600000 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f344a2f-4015-4626-a5de-2dc8421f10e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:46.657980 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0f6fd7-eabd-4171-b6df-3c79970a555b None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:46.671918 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f344a2f-4015-4626-a5de-2dc8421f10e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:46.673391 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f344a2f-4015-4626-a5de-2dc8421f10e0 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:46.692087 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d925b23-14e7-4037-8519-f2bb1de2efb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:46.692677 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d925b23-14e7-4037-8519-f2bb1de2efb5 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:46.692965 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d925b23-14e7-4037-8519-f2bb1de2efb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:46.693239 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d925b23-14e7-4037-8519-f2bb1de2efb5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:46.699763 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d925b23-14e7-4037-8519-f2bb1de2efb5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:46.702218 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d925b23-14e7-4037-8519-f2bb1de2efb5 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:46.713105 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-0d925b23-14e7-4037-8519-f2bb1de2efb5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 21:51:46.713105 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:46.713105 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:46.713105 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:46.713105 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:46.713105 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:46.713105 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:46.713105 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:46.713105 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:46.713105 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:46.713105 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:46.713105 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:46.713105 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 21:51:46.713105 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 21:51:46.713105 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 21:51:46.713998 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 21:51:46.713998 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:46.713998 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:46.713998 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:46.713998 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:46.713998 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:46.713998 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:46.713998 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:46.713998 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:46.713998 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:46.713998 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:46.713998 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:46.713998 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:46.713998 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:46.713998 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:46.713998 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:46.713998 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:46.714914 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 21:51:46.714914 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 21:51:46.714914 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 21:51:46.714914 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 21:51:46.714914 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 21:51:46.714914 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 21:51:46.714914 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 21:51:46.714914 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:46.714914 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 530/1064] 10.222.0.18 () {56 vars in 1111 bytes} [Sat Aug 6 21:51:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 21:51:46.715575 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0201ab9c-5730-4e84-b5a3-9a3e4dbb88c1 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:46.715849 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0201ab9c-5730-4e84-b5a3-9a3e4dbb88c1 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:46.716736 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0201ab9c-5730-4e84-b5a3-9a3e4dbb88c1 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:46.718015 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 531/1065] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:51:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:51:46.720609 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0201ab9c-5730-4e84-b5a3-9a3e4dbb88c1 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:46.771683 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:46.775840 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 535/1066] 10.222.0.18 () {58 vars in 1282 bytes} [Sat Aug 6 21:51:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 06 21:51:46.777070 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edc4bb5-a46a-479d-958d-d0da0d0490d2 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:46.777348 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edc4bb5-a46a-479d-958d-d0da0d0490d2 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:46.777616 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edc4bb5-a46a-479d-958d-d0da0d0490d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:46.788440 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:46.795449 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-8edc4bb5-a46a-479d-958d-d0da0d0490d2 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:51:46.796665 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-8edc4bb5-a46a-479d-958d-d0da0d0490d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:51:46.830316 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0201ab9c-5730-4e84-b5a3-9a3e4dbb88c1 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:46.863670 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 536/1067] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:51:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 21:51:46.864150 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e507a8-5a51-4f66-a6fe-97b652034176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:46.864712 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edc4bb5-a46a-479d-958d-d0da0d0490d2 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:46.864876 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e507a8-5a51-4f66-a6fe-97b652034176 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:46.865044 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e507a8-5a51-4f66-a6fe-97b652034176 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:46.865311 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e507a8-5a51-4f66-a6fe-97b652034176 None admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:46.865994 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e507a8-5a51-4f66-a6fe-97b652034176 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:46.866290 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e507a8-5a51-4f66-a6fe-97b652034176 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:46.868391 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e507a8-5a51-4f66-a6fe-97b652034176 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:46.883640 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 532/1068] 10.222.0.18 () {56 vars in 1148 bytes} [Sat Aug 6 21:51:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:51:46.889361 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e507a8-5a51-4f66-a6fe-97b652034176 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:46.961789 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edc4bb5-a46a-479d-958d-d0da0d0490d2 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:46.987701 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e507a8-5a51-4f66-a6fe-97b652034176 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:46.988714 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:46.997156 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efecf9d7-98a8-442c-8da7-8fc6b7e06d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:47.000553 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efecf9d7-98a8-442c-8da7-8fc6b7e06d7f None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:47.000553 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efecf9d7-98a8-442c-8da7-8fc6b7e06d7f None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:47.000553 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efecf9d7-98a8-442c-8da7-8fc6b7e06d7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:47.006922 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:47.008315 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efecf9d7-98a8-442c-8da7-8fc6b7e06d7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:47.009781 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efecf9d7-98a8-442c-8da7-8fc6b7e06d7f None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:47.020916 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106117f1-1b99-4e85-85ad-094d99976851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:47.021561 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106117f1-1b99-4e85-85ad-094d99976851 None admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:47.021834 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106117f1-1b99-4e85-85ad-094d99976851 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:47.022104 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106117f1-1b99-4e85-85ad-094d99976851 None admin] PATH_INFO: `/v3/projects/4bb6307eb0644d85a1d1daaa61352697/users/2a1319d33e9144859c6010843be312a3/roles/08bf738879a941dab8ac6323043dc991` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:47.024209 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106117f1-1b99-4e85-85ad-094d99976851 None admin] RBAC: Authorizing `identity:create_grant(project_id=4bb6307eb0644d85a1d1daaa61352697, user_id=2a1319d33e9144859c6010843be312a3, role_id=08bf738879a941dab8ac6323043dc991)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:47.024363 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-efecf9d7-98a8-442c-8da7-8fc6b7e06d7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:51:47.024363 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:47.024363 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:47.024363 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:47.024363 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:47.024363 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:47.024363 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:47.024363 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:47.024363 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:47.024363 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:47.024363 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:47.024363 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:47.024363 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:51:47.024363 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:51:47.024363 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:51:47.024363 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:51:47.025553 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:47.025553 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:47.025553 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:47.025553 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:47.025553 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:47.025553 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:47.025553 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:47.025553 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:47.025553 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:47.025553 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:47.025553 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:47.025553 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:47.025553 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:47.025553 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:47.025553 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:47.025553 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:47.025553 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:51:47.026675 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:51:47.026675 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:51:47.026675 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:47.026848 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106117f1-1b99-4e85-85ad-094d99976851 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:47.026918 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 537/1069] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:51:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:47.026918 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efecf9d7-98a8-442c-8da7-8fc6b7e06d7f None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:47.091009 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 533/1070] 10.222.0.18 () {58 vars in 1439 bytes} [Sat Aug 6 21:51:46 2022] PUT /identity/v3/projects/4bb6307eb0644d85a1d1daaa61352697/users/2a1319d33e9144859c6010843be312a3/roles/08bf738879a941dab8ac6323043dc991 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 21:51:47.095985 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106117f1-1b99-4e85-85ad-094d99976851 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:47.140759 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efecf9d7-98a8-442c-8da7-8fc6b7e06d7f None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:47.179280 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82a5ac3-4006-4d5a-b9ba-b580433b3f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:47.179927 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82a5ac3-4006-4d5a-b9ba-b580433b3f5a None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:47.180202 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82a5ac3-4006-4d5a-b9ba-b580433b3f5a None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:47.180511 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82a5ac3-4006-4d5a-b9ba-b580433b3f5a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:47.192819 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82a5ac3-4006-4d5a-b9ba-b580433b3f5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:51:47.197619 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-e82a5ac3-4006-4d5a-b9ba-b580433b3f5a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 21:51:47.197619 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:47.197619 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:47.197619 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:47.197619 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:47.197619 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:47.197619 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:47.197619 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:47.197619 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:47.197619 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:47.197619 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:47.197619 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:47.197619 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:51:47.197619 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:51:47.197619 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:51:47.197619 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:51:47.198782 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:51:47.198782 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:51:47.198782 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:51:47.198782 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:51:47.198782 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:47.198782 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:47.198782 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:47.198782 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:47.198782 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:47.198782 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:47.198782 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:47.198782 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:47.198782 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:47.198782 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:47.198782 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:47.198782 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:47.199840 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:47.199840 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:47.199840 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:47.199840 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:47.199840 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:51:47.199840 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:51:47.199840 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:51:47.199840 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:51:47.199840 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:51:47.199840 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:51:47.199840 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 21:51:47.199840 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:47.199840 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 538/1071] 10.222.0.18 () {56 vars in 1096 bytes} [Sat Aug 6 21:51:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:47.207329 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82a5ac3-4006-4d5a-b9ba-b580433b3f5a None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:47.240859 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106117f1-1b99-4e85-85ad-094d99976851 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:47.259396 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:47.286013 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a88ad45-75d1-492e-9456-d7345aed51a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:47.286731 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a88ad45-75d1-492e-9456-d7345aed51a8 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:47.287065 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a88ad45-75d1-492e-9456-d7345aed51a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:47.287396 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a88ad45-75d1-492e-9456-d7345aed51a8 None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:47.288040 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a88ad45-75d1-492e-9456-d7345aed51a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:47.288344 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a88ad45-75d1-492e-9456-d7345aed51a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:47.289741 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a88ad45-75d1-492e-9456-d7345aed51a8 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:47.302866 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 534/1072] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:51:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:47.323734 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82a5ac3-4006-4d5a-b9ba-b580433b3f5a None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:47.362652 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ae6129-4ec4-4ab5-ae52-6e99cab5de9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:47.363280 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ae6129-4ec4-4ab5-ae52-6e99cab5de9e None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:47.363559 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ae6129-4ec4-4ab5-ae52-6e99cab5de9e None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:47.363833 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ae6129-4ec4-4ab5-ae52-6e99cab5de9e None admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:47.364552 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ae6129-4ec4-4ab5-ae52-6e99cab5de9e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:47.364854 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ae6129-4ec4-4ab5-ae52-6e99cab5de9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:47.366264 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ae6129-4ec4-4ab5-ae52-6e99cab5de9e None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:47.377447 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 539/1073] 10.222.0.18 () {56 vars in 1099 bytes} [Sat Aug 6 21:51:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:51:47.383309 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a88ad45-75d1-492e-9456-d7345aed51a8 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:47.432204 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:47.498377 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a88ad45-75d1-492e-9456-d7345aed51a8 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:47.533169 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1ff539-afe2-449f-94ec-b70a26bb9b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:47.533799 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ff539-afe2-449f-94ec-b70a26bb9b2e None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:47.534076 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ff539-afe2-449f-94ec-b70a26bb9b2e None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:47.534347 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ff539-afe2-449f-94ec-b70a26bb9b2e None admin] PATH_INFO: `/v3/users/b066a3fc91654b4695624df25879ec29` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:47.548654 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1ff539-afe2-449f-94ec-b70a26bb9b2e None admin] RBAC: Authorizing `identity:get_user(user_id=b066a3fc91654b4695624df25879ec29)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:47.550185 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1ff539-afe2-449f-94ec-b70a26bb9b2e None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:47.551834 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 535/1074] 10.222.0.18 () {56 vars in 1177 bytes} [Sat Aug 6 21:51:47 2022] GET /identity/v3/users/b066a3fc91654b4695624df25879ec29 => generated 285 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:51:47.557760 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ae6129-4ec4-4ab5-ae52-6e99cab5de9e None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:47.666171 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ae6129-4ec4-4ab5-ae52-6e99cab5de9e None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:47.667664 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:47.698749 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b1cfd3-aa0d-4357-9323-4ea9372778b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:47.699342 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b1cfd3-aa0d-4357-9323-4ea9372778b2 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:47.699621 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b1cfd3-aa0d-4357-9323-4ea9372778b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:47.699893 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b1cfd3-aa0d-4357-9323-4ea9372778b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:47.766497 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b1cfd3-aa0d-4357-9323-4ea9372778b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:47.767851 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b1cfd3-aa0d-4357-9323-4ea9372778b2 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:47.863502 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 540/1075] 10.222.0.18 () {58 vars in 1282 bytes} [Sat Aug 6 21:51:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 21:51:47.869351 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1ff539-afe2-449f-94ec-b70a26bb9b2e None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:47.964480 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:47.968267 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1ff539-afe2-449f-94ec-b70a26bb9b2e None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:48.002167 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ed4132-e329-4582-9a2c-6715eef269d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:48.002785 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ed4132-e329-4582-9a2c-6715eef269d2 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:48.003063 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ed4132-e329-4582-9a2c-6715eef269d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:48.003336 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ed4132-e329-4582-9a2c-6715eef269d2 None admin] PATH_INFO: `/v3/projects/2d15f49835ea4bcbb21c7d5a53058393` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:48.004462 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ed4132-e329-4582-9a2c-6715eef269d2 None admin] RBAC: Authorizing `identity:get_project(project_id=2d15f49835ea4bcbb21c7d5a53058393)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:48.005895 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ed4132-e329-4582-9a2c-6715eef269d2 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:48.007496 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 536/1076] 10.222.0.18 () {56 vars in 1186 bytes} [Sat Aug 6 21:51:47 2022] GET /identity/v3/projects/2d15f49835ea4bcbb21c7d5a53058393 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:51:48.014399 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b1cfd3-aa0d-4357-9323-4ea9372778b2 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:48.016614 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:48.112488 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b1cfd3-aa0d-4357-9323-4ea9372778b2 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:48.146873 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0156d41-6ec0-4467-9698-79f4603cf86e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:48.147510 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0156d41-6ec0-4467-9698-79f4603cf86e None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:48.147735 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0156d41-6ec0-4467-9698-79f4603cf86e None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:48.148005 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0156d41-6ec0-4467-9698-79f4603cf86e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:48.148758 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0156d41-6ec0-4467-9698-79f4603cf86e None admin] RBAC: Adding query filter params (role.id=4b8dcf0491fd4072acc69fd894bf77d0, scope.project.id=2d15f49835ea4bcbb21c7d5a53058393, user.id=b066a3fc91654b4695624df25879ec29) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:48.149066 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0156d41-6ec0-4467-9698-79f4603cf86e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:48.150491 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0156d41-6ec0-4467-9698-79f4603cf86e None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:48.156236 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 541/1077] 10.222.0.18 () {56 vars in 1374 bytes} [Sat Aug 6 21:51:48 2022] GET /identity/v3/role_assignments?user.id=b066a3fc91654b4695624df25879ec29&scope.project.id=2d15f49835ea4bcbb21c7d5a53058393&role.id=4b8dcf0491fd4072acc69fd894bf77d0 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:51:49.113602 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ce26a-fabe-4f8d-a897-ff277140674d None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:49.114133 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ce26a-fabe-4f8d-a897-ff277140674d None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:49.114133 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ce26a-fabe-4f8d-a897-ff277140674d None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:49.115606 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 537/1078] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:51:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:51:49.120970 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065b3fc5-5ba7-4962-8d82-ded07ffbec3a None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:49.121255 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065b3fc5-5ba7-4962-8d82-ded07ffbec3a None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:49.121528 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065b3fc5-5ba7-4962-8d82-ded07ffbec3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:49.140330 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-065b3fc5-5ba7-4962-8d82-ded07ffbec3a None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:51:49.141579 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-065b3fc5-5ba7-4962-8d82-ded07ffbec3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:51:49.198967 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:49.207066 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 542/1079] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:51:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 21:51:49.229985 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696ce26a-fabe-4f8d-a897-ff277140674d None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:49.333026 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696ce26a-fabe-4f8d-a897-ff277140674d None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:49.370863 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc232a5-957f-4caf-866a-b79cbb13e06f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:49.371413 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc232a5-957f-4caf-866a-b79cbb13e06f None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:49.371686 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc232a5-957f-4caf-866a-b79cbb13e06f None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:49.371951 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc232a5-957f-4caf-866a-b79cbb13e06f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:49.381964 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc232a5-957f-4caf-866a-b79cbb13e06f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:51:49.386512 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-0fc232a5-957f-4caf-866a-b79cbb13e06f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 06 21:51:49.386512 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:49.386512 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:49.386512 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:49.386512 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:49.386512 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:49.386512 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:49.386512 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:49.386512 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:49.386512 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:49.386512 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:49.386512 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:49.386512 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:51:49.386512 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:51:49.386512 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:51:49.386512 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:51:49.387634 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:51:49.387634 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:51:49.387634 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:51:49.387634 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:51:49.387634 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:49.387634 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:49.387634 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:49.387634 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:49.387634 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:49.387634 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:49.387634 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:49.387634 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:49.387634 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:49.387634 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:49.387634 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:49.387634 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:49.388742 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:49.388742 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:49.388742 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:49.388742 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:49.388742 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:51:49.388742 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:51:49.388742 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:51:49.388742 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:51:49.388742 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:51:49.388742 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:51:49.388742 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 06 21:51:49.388742 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:49.392660 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 538/1080] 10.222.0.18 () {56 vars in 1120 bytes} [Sat Aug 6 21:51:49 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:49.393865 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065b3fc5-5ba7-4962-8d82-ded07ffbec3a None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:49.491286 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065b3fc5-5ba7-4962-8d82-ded07ffbec3a None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:49.524485 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be279499-10e2-4cca-bedc-896bfefe92d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:49.525112 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be279499-10e2-4cca-bedc-896bfefe92d0 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:49.525390 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be279499-10e2-4cca-bedc-896bfefe92d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:49.525660 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be279499-10e2-4cca-bedc-896bfefe92d0 None admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:49.526320 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be279499-10e2-4cca-bedc-896bfefe92d0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:49.526620 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be279499-10e2-4cca-bedc-896bfefe92d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:49.527977 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be279499-10e2-4cca-bedc-896bfefe92d0 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:49.536247 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 543/1081] 10.222.0.18 () {56 vars in 1115 bytes} [Sat Aug 6 21:51:49 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:49.541933 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc232a5-957f-4caf-866a-b79cbb13e06f None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:49.569834 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:49.640798 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc232a5-957f-4caf-866a-b79cbb13e06f None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:49.673907 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d273c7a7-31f7-4fb2-8231-852cc1914473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:49.674531 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d273c7a7-31f7-4fb2-8231-852cc1914473 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:49.674804 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d273c7a7-31f7-4fb2-8231-852cc1914473 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:49.675071 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d273c7a7-31f7-4fb2-8231-852cc1914473 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:49.685322 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d273c7a7-31f7-4fb2-8231-852cc1914473 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:49.686754 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d273c7a7-31f7-4fb2-8231-852cc1914473 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:49.701353 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-d273c7a7-31f7-4fb2-8231-852cc1914473 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:51:49.701353 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:49.701353 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:49.701353 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:49.701353 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:49.701353 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:49.701353 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:49.701353 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:49.701353 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:49.701353 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:49.701353 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:49.701353 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:49.701353 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:51:49.701353 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:51:49.701353 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:51:49.701353 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:51:49.707107 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:49.707107 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:49.707107 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:49.707107 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:49.707107 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:49.707107 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:49.707107 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:49.707107 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:49.707107 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:49.707107 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:49.707107 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:49.707107 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:49.707107 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:49.707107 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:49.707107 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:49.707107 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:49.707107 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:51:49.708067 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:51:49.708067 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:51:49.708067 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:49.708067 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 539/1082] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:51:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:49.708312 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be279499-10e2-4cca-bedc-896bfefe92d0 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:49.765614 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:49.806588 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be279499-10e2-4cca-bedc-896bfefe92d0 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:49.839934 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f10e44-8231-4775-b346-c664d82b52d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:49.840571 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f10e44-8231-4775-b346-c664d82b52d7 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:49.840849 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f10e44-8231-4775-b346-c664d82b52d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:49.841161 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f10e44-8231-4775-b346-c664d82b52d7 None admin] PATH_INFO: `/v3/domains` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:49.841799 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f10e44-8231-4775-b346-c664d82b52d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:49.842097 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f10e44-8231-4775-b346-c664d82b52d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:49.843446 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f10e44-8231-4775-b346-c664d82b52d7 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:49.854986 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 544/1083] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:51:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:49.861533 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d273c7a7-31f7-4fb2-8231-852cc1914473 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:49.938969 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:49.959300 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d273c7a7-31f7-4fb2-8231-852cc1914473 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:49.992339 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c046a0-0485-48de-b691-c75867090cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:49.992979 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c046a0-0485-48de-b691-c75867090cfd None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:49.993252 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c046a0-0485-48de-b691-c75867090cfd None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:49.993518 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c046a0-0485-48de-b691-c75867090cfd None admin] PATH_INFO: `/v3/users/nova` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:50.000926 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c046a0-0485-48de-b691-c75867090cfd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:50.002369 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c046a0-0485-48de-b691-c75867090cfd None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:50.013665 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-64c046a0-0485-48de-b691-c75867090cfd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 06 21:51:50.013665 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:50.013665 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:50.013665 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:50.013665 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:50.013665 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:50.013665 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:50.013665 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:50.013665 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:50.013665 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:50.013665 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:50.013665 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:50.013665 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 21:51:50.013665 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 21:51:50.013665 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 21:51:50.013665 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 21:51:50.014781 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:50.014781 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:50.014781 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 21:51:50.014781 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 21:51:50.014781 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 21:51:50.014781 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 21:51:50.014781 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:50.014781 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:50.014781 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:50.014781 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:50.014781 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:50.014781 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:50.014781 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:50.014781 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:50.014781 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:50.014781 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:50.014781 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:50.014781 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:50.016077 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:50.016077 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:50.016077 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 21:51:50.016077 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 21:51:50.016077 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 21:51:50.016077 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 21:51:50.016077 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 21:51:50.016077 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 21:51:50.016077 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 06 21:51:50.016077 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:50.019470 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 540/1084] 10.222.0.18 () {56 vars in 1093 bytes} [Sat Aug 6 21:51:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 21:51:50.020446 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f10e44-8231-4775-b346-c664d82b52d7 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:50.116484 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f10e44-8231-4775-b346-c664d82b52d7 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:50.134167 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:50.146420 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd1100e-c5a8-40d0-b5c9-fd2e8ef0337d None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:50.146955 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd1100e-c5a8-40d0-b5c9-fd2e8ef0337d None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:50.147041 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd1100e-c5a8-40d0-b5c9-fd2e8ef0337d None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:50.148335 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 541/1085] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:51:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:51:50.151450 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09288680-ce76-4201-b24d-ddb18ae15aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:50.152078 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09288680-ce76-4201-b24d-ddb18ae15aa7 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:50.152353 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09288680-ce76-4201-b24d-ddb18ae15aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:50.152664 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09288680-ce76-4201-b24d-ddb18ae15aa7 None admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:50.153202 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96a8c05-d15b-4045-9dbe-7e741c014ef2 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:50.153414 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09288680-ce76-4201-b24d-ddb18ae15aa7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:50.153531 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96a8c05-d15b-4045-9dbe-7e741c014ef2 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:50.153715 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09288680-ce76-4201-b24d-ddb18ae15aa7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:50.153843 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96a8c05-d15b-4045-9dbe-7e741c014ef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:50.155290 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09288680-ce76-4201-b24d-ddb18ae15aa7 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:50.171657 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-a96a8c05-d15b-4045-9dbe-7e741c014ef2 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:51:50.172881 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-a96a8c05-d15b-4045-9dbe-7e741c014ef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:51:50.173007 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 545/1086] 10.222.0.18 () {56 vars in 1133 bytes} [Sat Aug 6 21:51:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:50.178775 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09288680-ce76-4201-b24d-ddb18ae15aa7 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:50.240804 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 542/1087] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:51:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 21:51:50.263609 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96a8c05-d15b-4045-9dbe-7e741c014ef2 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:50.277259 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09288680-ce76-4201-b24d-ddb18ae15aa7 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:50.311898 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110acbd3-382f-486b-bc26-8d084578bf0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:50.312552 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110acbd3-382f-486b-bc26-8d084578bf0d None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:50.312829 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110acbd3-382f-486b-bc26-8d084578bf0d None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:50.313102 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110acbd3-382f-486b-bc26-8d084578bf0d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:50.329492 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:50.358860 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:50.367714 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96a8c05-d15b-4045-9dbe-7e741c014ef2 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:50.382514 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110acbd3-382f-486b-bc26-8d084578bf0d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:50.384017 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110acbd3-382f-486b-bc26-8d084578bf0d None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:50.402513 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0410669-be33-4ad5-ab88-f4ba3f2e27ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:50.403259 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0410669-be33-4ad5-ab88-f4ba3f2e27ec None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:50.404029 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0410669-be33-4ad5-ab88-f4ba3f2e27ec None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:50.404304 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0410669-be33-4ad5-ab88-f4ba3f2e27ec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:50.414613 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0410669-be33-4ad5-ab88-f4ba3f2e27ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:51:50.419849 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-e0410669-be33-4ad5-ab88-f4ba3f2e27ec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 21:51:50.419849 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:50.419849 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:50.419849 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:50.419849 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:50.419849 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:50.419849 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:50.419849 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:50.419849 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:50.419849 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:50.419849 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:50.419849 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:50.419849 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:51:50.419849 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:51:50.419849 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:51:50.419849 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:51:50.421013 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:51:50.421013 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:51:50.421013 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:51:50.421013 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:51:50.421013 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:50.421013 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:50.421013 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:50.421013 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:50.421013 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:50.421013 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:50.421013 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:50.421013 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:50.421013 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:50.421013 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:50.421013 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:50.421013 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:50.421876 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:50.421876 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:50.421876 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:50.421876 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:50.421876 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:51:50.421876 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:51:50.421876 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:51:50.421876 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:51:50.421876 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:51:50.421876 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:51:50.421876 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 21:51:50.421876 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:51:50.421876 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 543/1088] 10.222.0.18 () {56 vars in 1096 bytes} [Sat Aug 6 21:51:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 21:51:50.426843 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0410669-be33-4ad5-ab88-f4ba3f2e27ec None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:50.485024 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 546/1089] 10.222.0.18 () {58 vars in 1282 bytes} [Sat Aug 6 21:51:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 06 21:51:50.490884 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110acbd3-382f-486b-bc26-8d084578bf0d None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:50.526597 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0410669-be33-4ad5-ab88-f4ba3f2e27ec None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:50.552324 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:50.560387 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efa2912-88be-4fb0-8692-c61eea51d00c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:50.563590 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efa2912-88be-4fb0-8692-c61eea51d00c None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:50.563590 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efa2912-88be-4fb0-8692-c61eea51d00c None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:50.563590 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efa2912-88be-4fb0-8692-c61eea51d00c None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:50.563590 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efa2912-88be-4fb0-8692-c61eea51d00c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:50.563590 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efa2912-88be-4fb0-8692-c61eea51d00c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:50.563958 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efa2912-88be-4fb0-8692-c61eea51d00c None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:50.565557 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:50.572038 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 544/1090] 10.222.0.18 () {56 vars in 1099 bytes} [Sat Aug 6 21:51:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:51:50.578030 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efa2912-88be-4fb0-8692-c61eea51d00c None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:50.587641 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110acbd3-382f-486b-bc26-8d084578bf0d None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:50.623065 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95914c8-d523-4ee2-a742-8edaa67bfee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:50.623688 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95914c8-d523-4ee2-a742-8edaa67bfee6 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:50.623961 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95914c8-d523-4ee2-a742-8edaa67bfee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:50.624229 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95914c8-d523-4ee2-a742-8edaa67bfee6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:50.633992 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95914c8-d523-4ee2-a742-8edaa67bfee6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:50.635445 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95914c8-d523-4ee2-a742-8edaa67bfee6 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:50.649884 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-a95914c8-d523-4ee2-a742-8edaa67bfee6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:51:50.649884 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:50.649884 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:50.649884 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:50.649884 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:50.649884 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:50.649884 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:50.649884 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:50.649884 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:50.649884 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:50.649884 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:50.649884 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:50.649884 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:51:50.649884 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:51:50.649884 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:51:50.649884 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:51:50.650843 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:50.650843 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:50.650843 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:50.650843 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:50.650843 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:50.650843 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:50.650843 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:50.650843 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:50.650843 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:50.650843 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:50.650843 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:50.650843 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:50.650843 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:50.650843 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:50.650843 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:50.650843 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:50.650843 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:51:50.651828 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:51:50.651828 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:51:50.651828 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:50.651828 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 547/1091] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:51:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:50.657406 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95914c8-d523-4ee2-a742-8edaa67bfee6 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:50.676115 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efa2912-88be-4fb0-8692-c61eea51d00c None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:50.710371 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a6e73d-6d64-4074-a67a-f83a9f125de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:50.710999 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a6e73d-6d64-4074-a67a-f83a9f125de8 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:50.711280 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a6e73d-6d64-4074-a67a-f83a9f125de8 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:50.711553 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a6e73d-6d64-4074-a67a-f83a9f125de8 None admin] PATH_INFO: `/v3/users/b066a3fc91654b4695624df25879ec29` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:50.712720 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a6e73d-6d64-4074-a67a-f83a9f125de8 None admin] RBAC: Authorizing `identity:get_user(user_id=b066a3fc91654b4695624df25879ec29)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:50.714165 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a6e73d-6d64-4074-a67a-f83a9f125de8 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:50.721043 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 545/1092] 10.222.0.18 () {56 vars in 1177 bytes} [Sat Aug 6 21:51:50 2022] GET /identity/v3/users/b066a3fc91654b4695624df25879ec29 => generated 285 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:51:50.722197 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a6e73d-6d64-4074-a67a-f83a9f125de8 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:50.754259 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95914c8-d523-4ee2-a742-8edaa67bfee6 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:50.778089 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:50.780148 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:50.788049 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7e8580-b8bf-4abf-832e-a62ce7eedd87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:50.788678 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7e8580-b8bf-4abf-832e-a62ce7eedd87 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:50.788956 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7e8580-b8bf-4abf-832e-a62ce7eedd87 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:50.789286 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7e8580-b8bf-4abf-832e-a62ce7eedd87 None admin] PATH_INFO: `/v3/domains` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:50.789934 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7e8580-b8bf-4abf-832e-a62ce7eedd87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:50.790237 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7e8580-b8bf-4abf-832e-a62ce7eedd87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:50.791590 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7e8580-b8bf-4abf-832e-a62ce7eedd87 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:50.803253 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 548/1093] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:51:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:51:50.808765 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7e8580-b8bf-4abf-832e-a62ce7eedd87 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:50.822236 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a6e73d-6d64-4074-a67a-f83a9f125de8 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:50.858467 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d501aaf4-285c-4e8f-824b-fe9f73d35759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:50.859085 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d501aaf4-285c-4e8f-824b-fe9f73d35759 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:50.859370 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d501aaf4-285c-4e8f-824b-fe9f73d35759 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:50.859648 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d501aaf4-285c-4e8f-824b-fe9f73d35759 None admin] PATH_INFO: `/v3/projects/2d15f49835ea4bcbb21c7d5a53058393` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:50.860798 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d501aaf4-285c-4e8f-824b-fe9f73d35759 None admin] RBAC: Authorizing `identity:get_project(project_id=2d15f49835ea4bcbb21c7d5a53058393)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:50.862232 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d501aaf4-285c-4e8f-824b-fe9f73d35759 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:50.863852 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 546/1094] 10.222.0.18 () {56 vars in 1186 bytes} [Sat Aug 6 21:51:50 2022] GET /identity/v3/projects/2d15f49835ea4bcbb21c7d5a53058393 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:51:50.869576 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d501aaf4-285c-4e8f-824b-fe9f73d35759 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:50.910284 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7e8580-b8bf-4abf-832e-a62ce7eedd87 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:50.944736 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e856c511-d86a-4e9d-95b7-03643bda30d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:50.945358 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e856c511-d86a-4e9d-95b7-03643bda30d8 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:50.945633 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e856c511-d86a-4e9d-95b7-03643bda30d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:50.945900 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e856c511-d86a-4e9d-95b7-03643bda30d8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:50.952744 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e856c511-d86a-4e9d-95b7-03643bda30d8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:50.954159 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e856c511-d86a-4e9d-95b7-03643bda30d8 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:50.969347 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d501aaf4-285c-4e8f-824b-fe9f73d35759 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:50.970394 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-e856c511-d86a-4e9d-95b7-03643bda30d8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 21:51:50.970394 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:50.970394 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:50.970394 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:50.970394 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:50.970394 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:50.970394 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:50.970394 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:50.970394 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:50.970394 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:50.970394 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:50.970394 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:50.970394 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 21:51:50.970394 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 21:51:50.970394 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 21:51:50.971346 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 21:51:50.971346 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:50.971346 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:50.971346 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:50.971346 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:50.971346 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:50.971346 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:50.971346 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:50.971346 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:50.971346 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:50.971346 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:50.971346 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:50.971346 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:50.971346 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:50.971346 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:50.971346 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:50.971346 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:50.972237 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 21:51:50.972237 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 21:51:50.972237 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 21:51:50.972237 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 21:51:50.972237 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 21:51:50.972237 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 21:51:50.972237 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 21:51:50.972237 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:50.972237 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 549/1095] 10.222.0.18 () {56 vars in 1111 bytes} [Sat Aug 6 21:51:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 21:51:50.977165 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e856c511-d86a-4e9d-95b7-03643bda30d8 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:50.996980 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:51.005959 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c79b17-4b88-440b-8aad-1ee321cc7b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:51.006554 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c79b17-4b88-440b-8aad-1ee321cc7b7c None admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:51.006833 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c79b17-4b88-440b-8aad-1ee321cc7b7c None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:51.007101 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c79b17-4b88-440b-8aad-1ee321cc7b7c None admin] PATH_INFO: `/v3/projects/2d15f49835ea4bcbb21c7d5a53058393/users/b066a3fc91654b4695624df25879ec29/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:51.009184 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c79b17-4b88-440b-8aad-1ee321cc7b7c None admin] RBAC: Authorizing `identity:create_grant(project_id=2d15f49835ea4bcbb21c7d5a53058393, user_id=b066a3fc91654b4695624df25879ec29, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:51.010780 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c79b17-4b88-440b-8aad-1ee321cc7b7c None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:51.024417 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:51.072380 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e856c511-d86a-4e9d-95b7-03643bda30d8 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:51.104062 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91f989a-3a27-47da-8040-b6a36549f41f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:51.104671 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91f989a-3a27-47da-8040-b6a36549f41f None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:51.104948 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91f989a-3a27-47da-8040-b6a36549f41f None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:51.105217 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91f989a-3a27-47da-8040-b6a36549f41f None admin] PATH_INFO: `/v3/projects` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:51.105898 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91f989a-3a27-47da-8040-b6a36549f41f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:51.106199 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91f989a-3a27-47da-8040-b6a36549f41f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:51.108590 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91f989a-3a27-47da-8040-b6a36549f41f None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:51.114961 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 547/1096] 10.222.0.18 () {58 vars in 1439 bytes} [Sat Aug 6 21:51:50 2022] PUT /identity/v3/projects/2d15f49835ea4bcbb21c7d5a53058393/users/b066a3fc91654b4695624df25879ec29/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 21:51:51.121863 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 550/1097] 10.222.0.18 () {56 vars in 1148 bytes} [Sat Aug 6 21:51:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:51:51.128508 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c79b17-4b88-440b-8aad-1ee321cc7b7c None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:51.236709 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c79b17-4b88-440b-8aad-1ee321cc7b7c None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:51.273618 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c16ca9f-afc5-4662-8b15-784f5cca6f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:51.274165 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c16ca9f-afc5-4662-8b15-784f5cca6f1c None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:51.274428 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c16ca9f-afc5-4662-8b15-784f5cca6f1c None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:51.274699 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c16ca9f-afc5-4662-8b15-784f5cca6f1c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:51.275422 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c16ca9f-afc5-4662-8b15-784f5cca6f1c None admin] RBAC: Adding query filter params (role.id=08bf738879a941dab8ac6323043dc991, scope.project.id=4bb6307eb0644d85a1d1daaa61352697, user.id=2a1319d33e9144859c6010843be312a3) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:51.275858 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c16ca9f-afc5-4662-8b15-784f5cca6f1c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:51.277329 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c16ca9f-afc5-4662-8b15-784f5cca6f1c None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:51.283906 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 548/1098] 10.222.0.18 () {56 vars in 1374 bytes} [Sat Aug 6 21:51:51 2022] GET /identity/v3/role_assignments?user.id=2a1319d33e9144859c6010843be312a3&scope.project.id=4bb6307eb0644d85a1d1daaa61352697&role.id=08bf738879a941dab8ac6323043dc991 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:53.111430 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f4c9b5-9ef3-4a31-8d44-c3a2923fc84e None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:53.111961 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f4c9b5-9ef3-4a31-8d44-c3a2923fc84e None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:53.111961 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f4c9b5-9ef3-4a31-8d44-c3a2923fc84e None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:53.113337 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 551/1099] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:51:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:51:53.118388 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178341dc-870f-445c-a753-cd3b6e0781c9 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:53.118810 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178341dc-870f-445c-a753-cd3b6e0781c9 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:53.119187 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178341dc-870f-445c-a753-cd3b6e0781c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:53.139200 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-178341dc-870f-445c-a753-cd3b6e0781c9 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:51:53.140363 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-178341dc-870f-445c-a753-cd3b6e0781c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:51:53.184314 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:53.206045 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 549/1100] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:51:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 21:51:53.229005 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f4c9b5-9ef3-4a31-8d44-c3a2923fc84e None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:53.329542 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f4c9b5-9ef3-4a31-8d44-c3a2923fc84e None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:53.338392 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:53.363056 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dde4d20-b735-465e-99a6-ded4192f61b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:53.363639 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dde4d20-b735-465e-99a6-ded4192f61b2 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:53.364113 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dde4d20-b735-465e-99a6-ded4192f61b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:53.364621 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dde4d20-b735-465e-99a6-ded4192f61b2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:53.375455 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dde4d20-b735-465e-99a6-ded4192f61b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:51:53.380132 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-3dde4d20-b735-465e-99a6-ded4192f61b2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 21:51:53.380132 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:51:53.380132 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:51:53.380132 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:51:53.380132 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:51:53.380132 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:51:53.380132 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:51:53.380132 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:51:53.380132 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:51:53.380132 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:51:53.380132 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:51:53.380132 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:51:53.380132 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:51:53.380132 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:51:53.380132 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:51:53.380132 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:51:53.381271 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:51:53.381271 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:51:53.381271 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:51:53.381271 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:51:53.381271 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:51:53.381271 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:51:53.381271 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:51:53.381271 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:51:53.381271 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:51:53.381271 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:51:53.381271 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:51:53.381271 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:51:53.381271 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:51:53.381271 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:51:53.381271 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:51:53.381271 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:51:53.382278 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:51:53.382278 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:51:53.382278 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:51:53.382278 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:51:53.382278 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:51:53.382278 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:51:53.382278 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:51:53.382278 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:51:53.382278 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:51:53.382278 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:51:53.382278 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 21:51:53.382278 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:51:53.382278 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 552/1101] 10.222.0.18 () {56 vars in 1096 bytes} [Sat Aug 6 21:51:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:51:53.387419 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178341dc-870f-445c-a753-cd3b6e0781c9 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:53.484515 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178341dc-870f-445c-a753-cd3b6e0781c9 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:53.517185 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a50959d-03cd-4bac-98a2-cda38fb6f683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:53.517771 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a50959d-03cd-4bac-98a2-cda38fb6f683 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:53.518045 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a50959d-03cd-4bac-98a2-cda38fb6f683 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:53.518310 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a50959d-03cd-4bac-98a2-cda38fb6f683 None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:53.518954 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a50959d-03cd-4bac-98a2-cda38fb6f683 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:53.519246 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a50959d-03cd-4bac-98a2-cda38fb6f683 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:53.520623 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a50959d-03cd-4bac-98a2-cda38fb6f683 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:53.528293 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 550/1102] 10.222.0.18 () {56 vars in 1099 bytes} [Sat Aug 6 21:51:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:53.534180 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dde4d20-b735-465e-99a6-ded4192f61b2 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:53.634028 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dde4d20-b735-465e-99a6-ded4192f61b2 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:53.666954 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ef5390-f4f9-4b99-91c5-ce77be2079eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:53.667568 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ef5390-f4f9-4b99-91c5-ce77be2079eb None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:53.667849 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ef5390-f4f9-4b99-91c5-ce77be2079eb None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:53.668123 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ef5390-f4f9-4b99-91c5-ce77be2079eb None admin] PATH_INFO: `/v3/users/b066a3fc91654b4695624df25879ec29` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:53.670181 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ef5390-f4f9-4b99-91c5-ce77be2079eb None admin] RBAC: Authorizing `identity:get_user(user_id=b066a3fc91654b4695624df25879ec29)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:53.671636 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ef5390-f4f9-4b99-91c5-ce77be2079eb None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:53.673229 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 553/1103] 10.222.0.18 () {56 vars in 1177 bytes} [Sat Aug 6 21:51:53 2022] GET /identity/v3/users/b066a3fc91654b4695624df25879ec29 => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:53.678933 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a50959d-03cd-4bac-98a2-cda38fb6f683 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:53.721743 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:53.776771 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a50959d-03cd-4bac-98a2-cda38fb6f683 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:53.809213 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2a071a-5079-411b-9891-472aa0cf102b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:53.810477 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2a071a-5079-411b-9891-472aa0cf102b None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:53.810751 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2a071a-5079-411b-9891-472aa0cf102b None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:53.811018 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2a071a-5079-411b-9891-472aa0cf102b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:53.875611 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2a071a-5079-411b-9891-472aa0cf102b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:53.877088 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2a071a-5079-411b-9891-472aa0cf102b None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:53.916606 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:53.972911 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 551/1104] 10.222.0.18 () {58 vars in 1282 bytes} [Sat Aug 6 21:51:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 21:51:53.979004 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ef5390-f4f9-4b99-91c5-ce77be2079eb None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:54.013786 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:54.076590 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ef5390-f4f9-4b99-91c5-ce77be2079eb None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:54.108571 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78503e27-8203-4fca-9db4-2d5dd06b921c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:54.109168 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78503e27-8203-4fca-9db4-2d5dd06b921c None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:54.109442 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78503e27-8203-4fca-9db4-2d5dd06b921c None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:54.109709 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78503e27-8203-4fca-9db4-2d5dd06b921c None admin] PATH_INFO: `/v3/projects/2d15f49835ea4bcbb21c7d5a53058393` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:54.110804 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78503e27-8203-4fca-9db4-2d5dd06b921c None admin] RBAC: Authorizing `identity:get_project(project_id=2d15f49835ea4bcbb21c7d5a53058393)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:54.112222 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78503e27-8203-4fca-9db4-2d5dd06b921c None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:54.113860 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 554/1105] 10.222.0.18 () {56 vars in 1186 bytes} [Sat Aug 6 21:51:53 2022] GET /identity/v3/projects/2d15f49835ea4bcbb21c7d5a53058393 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:54.119932 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2a071a-5079-411b-9891-472aa0cf102b None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:54.215764 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2a071a-5079-411b-9891-472aa0cf102b None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:54.249215 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65ab2d5-44e3-4845-a4fa-f328c60fb5f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:54.249797 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65ab2d5-44e3-4845-a4fa-f328c60fb5f6 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:54.250073 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65ab2d5-44e3-4845-a4fa-f328c60fb5f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:54.250341 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65ab2d5-44e3-4845-a4fa-f328c60fb5f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:54.251053 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65ab2d5-44e3-4845-a4fa-f328c60fb5f6 None admin] RBAC: Adding query filter params (role.id=4b8dcf0491fd4072acc69fd894bf77d0, scope.project.id=2d15f49835ea4bcbb21c7d5a53058393, user.id=b066a3fc91654b4695624df25879ec29) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:54.251357 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65ab2d5-44e3-4845-a4fa-f328c60fb5f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:54.252796 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65ab2d5-44e3-4845-a4fa-f328c60fb5f6 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:54.258319 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 552/1106] 10.222.0.18 () {56 vars in 1374 bytes} [Sat Aug 6 21:51:54 2022] GET /identity/v3/role_assignments?user.id=b066a3fc91654b4695624df25879ec29&scope.project.id=2d15f49835ea4bcbb21c7d5a53058393&role.id=4b8dcf0491fd4072acc69fd894bf77d0 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:56.264714 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddffb50-62c5-46f5-b447-4f2767f7d0f1 None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:56.265270 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddffb50-62c5-46f5-b447-4f2767f7d0f1 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:56.265270 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddffb50-62c5-46f5-b447-4f2767f7d0f1 None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:56.266595 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 555/1107] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:51:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:51:56.271651 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367e00b9-dfb5-4efe-bcfc-8dbc026487bf None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:56.272014 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367e00b9-dfb5-4efe-bcfc-8dbc026487bf None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:56.272188 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367e00b9-dfb5-4efe-bcfc-8dbc026487bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:56.285546 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:56.291085 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-367e00b9-dfb5-4efe-bcfc-8dbc026487bf None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:51:56.292282 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-367e00b9-dfb5-4efe-bcfc-8dbc026487bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:51:56.359229 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 553/1108] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:51:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 21:51:56.383547 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddffb50-62c5-46f5-b447-4f2767f7d0f1 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:56.481810 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddffb50-62c5-46f5-b447-4f2767f7d0f1 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:56.484312 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:56.516434 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1da19cb-ac9a-4205-a074-e64da7f8abb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:56.516981 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1da19cb-ac9a-4205-a074-e64da7f8abb7 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:56.517256 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1da19cb-ac9a-4205-a074-e64da7f8abb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:56.517524 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1da19cb-ac9a-4205-a074-e64da7f8abb7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:56.518611 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1da19cb-ac9a-4205-a074-e64da7f8abb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:56.520027 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1da19cb-ac9a-4205-a074-e64da7f8abb7 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:56.521625 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 556/1109] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:51:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:56.527326 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367e00b9-dfb5-4efe-bcfc-8dbc026487bf None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:56.623037 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367e00b9-dfb5-4efe-bcfc-8dbc026487bf None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:56.656218 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8871144b-24e3-4185-954b-b7609b50e90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:56.656829 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8871144b-24e3-4185-954b-b7609b50e90b None admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:56.657103 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8871144b-24e3-4185-954b-b7609b50e90b None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:56.657375 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8871144b-24e3-4185-954b-b7609b50e90b None admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:56.658010 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8871144b-24e3-4185-954b-b7609b50e90b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:56.659420 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8871144b-24e3-4185-954b-b7609b50e90b None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:56.667878 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:56.833875 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 554/1110] 10.222.0.18 () {60 vars in 1132 bytes} [Sat Aug 6 21:51:56 2022] POST /identity/v3/users => generated 286 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 21:51:58.877226 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f8f4a3-ad5d-4bdd-b651-1048660c4ebb None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:58.877812 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f8f4a3-ad5d-4bdd-b651-1048660c4ebb None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:58.877812 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f8f4a3-ad5d-4bdd-b651-1048660c4ebb None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:58.879133 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 557/1111] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:51:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:51:58.884278 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a086fe-0be1-4433-8942-116f9e12f9c6 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:58.884712 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a086fe-0be1-4433-8942-116f9e12f9c6 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:58.884859 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a086fe-0be1-4433-8942-116f9e12f9c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:58.903673 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-98a086fe-0be1-4433-8942-116f9e12f9c6 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:51:58.904881 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-98a086fe-0be1-4433-8942-116f9e12f9c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:51:58.969294 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 555/1112] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:51:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 21:51:58.992227 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f8f4a3-ad5d-4bdd-b651-1048660c4ebb None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:59.091595 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f8f4a3-ad5d-4bdd-b651-1048660c4ebb None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:59.124702 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4981c42-5dda-40ce-a860-db967e3f0e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:59.125331 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4981c42-5dda-40ce-a860-db967e3f0e1b None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:59.125605 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4981c42-5dda-40ce-a860-db967e3f0e1b None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:59.125873 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4981c42-5dda-40ce-a860-db967e3f0e1b None admin] PATH_INFO: `/v3/roles/93c058ac49e242fd8517c80c552dea2d` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:59.127282 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4981c42-5dda-40ce-a860-db967e3f0e1b None admin] RBAC: Authorizing `identity:get_role(role_id=93c058ac49e242fd8517c80c552dea2d)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:59.128615 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4981c42-5dda-40ce-a860-db967e3f0e1b None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:59.129898 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 558/1113] 10.222.0.18 () {56 vars in 1177 bytes} [Sat Aug 6 21:51:58 2022] GET /identity/v3/roles/93c058ac49e242fd8517c80c552dea2d => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:59.135798 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a086fe-0be1-4433-8942-116f9e12f9c6 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:59.232731 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a086fe-0be1-4433-8942-116f9e12f9c6 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:59.254424 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:59.265244 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d230120-d67a-4158-b0ee-9271aa8fab8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:59.265860 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d230120-d67a-4158-b0ee-9271aa8fab8e None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:59.266137 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d230120-d67a-4158-b0ee-9271aa8fab8e None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:59.266409 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d230120-d67a-4158-b0ee-9271aa8fab8e None admin] PATH_INFO: `/v3/users/e21addffee5e444ca2fc47473f476b88` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:59.279527 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d230120-d67a-4158-b0ee-9271aa8fab8e None admin] RBAC: Authorizing `identity:get_user(user_id=e21addffee5e444ca2fc47473f476b88)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:59.281014 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d230120-d67a-4158-b0ee-9271aa8fab8e None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:59.282568 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 556/1114] 10.222.0.18 () {56 vars in 1177 bytes} [Sat Aug 6 21:51:59 2022] GET /identity/v3/users/e21addffee5e444ca2fc47473f476b88 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:59.288170 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4981c42-5dda-40ce-a860-db967e3f0e1b None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:59.330097 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:59.388905 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4981c42-5dda-40ce-a860-db967e3f0e1b None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:59.421939 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11218012-41ab-440b-af79-6a60e42431ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:59.422527 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11218012-41ab-440b-af79-6a60e42431ae None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:59.422800 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11218012-41ab-440b-af79-6a60e42431ae None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:59.423067 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11218012-41ab-440b-af79-6a60e42431ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:59.487049 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11218012-41ab-440b-af79-6a60e42431ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:59.488494 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11218012-41ab-440b-af79-6a60e42431ae None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:59.526541 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:51:59.582938 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 559/1115] 10.222.0.18 () {58 vars in 1282 bytes} [Sat Aug 6 21:51:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 21:51:59.588826 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d230120-d67a-4158-b0ee-9271aa8fab8e None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:59.689002 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d230120-d67a-4158-b0ee-9271aa8fab8e None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:59.723770 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548b307c-2623-442d-8f19-cbee4b4d945b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:59.724382 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548b307c-2623-442d-8f19-cbee4b4d945b None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:59.724692 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548b307c-2623-442d-8f19-cbee4b4d945b None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:59.724963 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548b307c-2623-442d-8f19-cbee4b4d945b None admin] PATH_INFO: `/v3/projects/2d15f49835ea4bcbb21c7d5a53058393` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:59.726063 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548b307c-2623-442d-8f19-cbee4b4d945b None admin] RBAC: Authorizing `identity:get_project(project_id=2d15f49835ea4bcbb21c7d5a53058393)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:59.727490 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548b307c-2623-442d-8f19-cbee4b4d945b None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:59.729142 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 557/1116] 10.222.0.18 () {56 vars in 1186 bytes} [Sat Aug 6 21:51:59 2022] GET /identity/v3/projects/2d15f49835ea4bcbb21c7d5a53058393 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:51:59.735674 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11218012-41ab-440b-af79-6a60e42431ae None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:51:59.832009 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11218012-41ab-440b-af79-6a60e42431ae None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:51:59.864638 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23a862a-5492-4f44-8204-85ae48e8113f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:51:59.865251 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23a862a-5492-4f44-8204-85ae48e8113f None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:51:59.865532 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23a862a-5492-4f44-8204-85ae48e8113f None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:51:59.865809 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23a862a-5492-4f44-8204-85ae48e8113f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:51:59.866534 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23a862a-5492-4f44-8204-85ae48e8113f None admin] RBAC: Adding query filter params (role.id=93c058ac49e242fd8517c80c552dea2d, scope.project.id=2d15f49835ea4bcbb21c7d5a53058393, user.id=e21addffee5e444ca2fc47473f476b88) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:51:59.866840 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23a862a-5492-4f44-8204-85ae48e8113f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:51:59.868247 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23a862a-5492-4f44-8204-85ae48e8113f None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:51:59.874034 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 560/1117] 10.222.0.18 () {56 vars in 1374 bytes} [Sat Aug 6 21:51:59 2022] GET /identity/v3/role_assignments?user.id=e21addffee5e444ca2fc47473f476b88&scope.project.id=2d15f49835ea4bcbb21c7d5a53058393&role.id=93c058ac49e242fd8517c80c552dea2d => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:52:01.832215 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd69785-2b4c-4d14-853b-c1cfcfe8b36e None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:01.832792 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd69785-2b4c-4d14-853b-c1cfcfe8b36e None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:01.832792 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd69785-2b4c-4d14-853b-c1cfcfe8b36e None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:01.834103 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 558/1118] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:52:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:52:01.839132 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548af6d4-6f35-47c7-9270-27039a3104d5 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:01.839548 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548af6d4-6f35-47c7-9270-27039a3104d5 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:01.839666 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548af6d4-6f35-47c7-9270-27039a3104d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:01.847416 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:52:01.858218 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-548af6d4-6f35-47c7-9270-27039a3104d5 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:52:01.859406 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-548af6d4-6f35-47c7-9270-27039a3104d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:52:01.924290 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 561/1119] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:52:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 21:52:01.946141 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd69785-2b4c-4d14-853b-c1cfcfe8b36e None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:52:01.995527 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:52:02.045731 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd69785-2b4c-4d14-853b-c1cfcfe8b36e None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:52:02.077786 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59f3880-0b69-4d17-ba46-b5e678121755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:52:02.078397 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59f3880-0b69-4d17-ba46-b5e678121755 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:02.078676 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59f3880-0b69-4d17-ba46-b5e678121755 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:02.078949 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59f3880-0b69-4d17-ba46-b5e678121755 None admin] PATH_INFO: `/v3/roles/93c058ac49e242fd8517c80c552dea2d` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:02.080312 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59f3880-0b69-4d17-ba46-b5e678121755 None admin] RBAC: Authorizing `identity:get_role(role_id=93c058ac49e242fd8517c80c552dea2d)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:52:02.081704 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59f3880-0b69-4d17-ba46-b5e678121755 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:52:02.083026 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 559/1120] 10.222.0.18 () {56 vars in 1177 bytes} [Sat Aug 6 21:52:01 2022] GET /identity/v3/roles/93c058ac49e242fd8517c80c552dea2d => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:52:02.088781 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548af6d4-6f35-47c7-9270-27039a3104d5 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:52:02.185689 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548af6d4-6f35-47c7-9270-27039a3104d5 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:52:02.218948 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcf5fbc-985d-4ad9-9454-2f467214ce2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:52:02.219565 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcf5fbc-985d-4ad9-9454-2f467214ce2d None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:02.219847 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcf5fbc-985d-4ad9-9454-2f467214ce2d None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:02.220121 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcf5fbc-985d-4ad9-9454-2f467214ce2d None admin] PATH_INFO: `/v3/users/e21addffee5e444ca2fc47473f476b88` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:02.221270 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcf5fbc-985d-4ad9-9454-2f467214ce2d None admin] RBAC: Authorizing `identity:get_user(user_id=e21addffee5e444ca2fc47473f476b88)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:52:02.222691 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcf5fbc-985d-4ad9-9454-2f467214ce2d None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:52:02.224247 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 562/1121] 10.222.0.18 () {56 vars in 1177 bytes} [Sat Aug 6 21:52:02 2022] GET /identity/v3/users/e21addffee5e444ca2fc47473f476b88 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:52:02.229950 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59f3880-0b69-4d17-ba46-b5e678121755 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:52:02.327488 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59f3880-0b69-4d17-ba46-b5e678121755 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:52:02.343634 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:52:02.361563 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3df8d1-f8b2-41e7-9d21-8c247ab75b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:52:02.362181 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3df8d1-f8b2-41e7-9d21-8c247ab75b52 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:02.362453 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3df8d1-f8b2-41e7-9d21-8c247ab75b52 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:02.362720 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3df8d1-f8b2-41e7-9d21-8c247ab75b52 None admin] PATH_INFO: `/v3/projects/2d15f49835ea4bcbb21c7d5a53058393` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:02.363844 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3df8d1-f8b2-41e7-9d21-8c247ab75b52 None admin] RBAC: Authorizing `identity:get_project(project_id=2d15f49835ea4bcbb21c7d5a53058393)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:52:02.365288 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3df8d1-f8b2-41e7-9d21-8c247ab75b52 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:52:02.366874 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 560/1122] 10.222.0.18 () {56 vars in 1186 bytes} [Sat Aug 6 21:52:02 2022] GET /identity/v3/projects/2d15f49835ea4bcbb21c7d5a53058393 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:52:02.372909 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcf5fbc-985d-4ad9-9454-2f467214ce2d None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:52:02.375724 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:52:02.471601 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcf5fbc-985d-4ad9-9454-2f467214ce2d None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:52:02.503604 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd862305-5531-47e8-b65f-f6a5ed91392f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:52:02.504245 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd862305-5531-47e8-b65f-f6a5ed91392f None admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:02.504558 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd862305-5531-47e8-b65f-f6a5ed91392f None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:02.504835 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd862305-5531-47e8-b65f-f6a5ed91392f None admin] PATH_INFO: `/v3/projects/2d15f49835ea4bcbb21c7d5a53058393/users/e21addffee5e444ca2fc47473f476b88/roles/93c058ac49e242fd8517c80c552dea2d` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:02.507033 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd862305-5531-47e8-b65f-f6a5ed91392f None admin] RBAC: Authorizing `identity:create_grant(project_id=2d15f49835ea4bcbb21c7d5a53058393, user_id=e21addffee5e444ca2fc47473f476b88, role_id=93c058ac49e242fd8517c80c552dea2d)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:52:02.508484 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd862305-5531-47e8-b65f-f6a5ed91392f None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:52:02.581557 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 563/1123] 10.222.0.18 () {58 vars in 1439 bytes} [Sat Aug 6 21:52:02 2022] PUT /identity/v3/projects/2d15f49835ea4bcbb21c7d5a53058393/users/e21addffee5e444ca2fc47473f476b88/roles/93c058ac49e242fd8517c80c552dea2d => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 21:52:04.572785 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5597faee-ea4c-4d62-857f-52c071a2b6ce None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:04.573324 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5597faee-ea4c-4d62-857f-52c071a2b6ce None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:04.573324 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5597faee-ea4c-4d62-857f-52c071a2b6ce None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:04.574744 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 561/1124] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:52:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:52:04.579899 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab65f2-ea1e-417d-bb5d-a1fb930381f9 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:04.580271 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab65f2-ea1e-417d-bb5d-a1fb930381f9 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:04.580484 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab65f2-ea1e-417d-bb5d-a1fb930381f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:04.598723 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-aaab65f2-ea1e-417d-bb5d-a1fb930381f9 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:52:04.599900 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-aaab65f2-ea1e-417d-bb5d-a1fb930381f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:52:04.656321 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:52:04.666240 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 564/1125] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:52:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 21:52:04.689774 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5597faee-ea4c-4d62-857f-52c071a2b6ce None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:52:04.788499 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5597faee-ea4c-4d62-857f-52c071a2b6ce None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:52:04.821460 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7bc1b3-9512-42b7-afda-644c0b0ebe39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:52:04.822072 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7bc1b3-9512-42b7-afda-644c0b0ebe39 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:04.822349 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7bc1b3-9512-42b7-afda-644c0b0ebe39 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:04.822619 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7bc1b3-9512-42b7-afda-644c0b0ebe39 None admin] PATH_INFO: `/v3/roles/93c058ac49e242fd8517c80c552dea2d` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:04.823980 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7bc1b3-9512-42b7-afda-644c0b0ebe39 None admin] RBAC: Authorizing `identity:get_role(role_id=93c058ac49e242fd8517c80c552dea2d)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:52:04.825962 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7bc1b3-9512-42b7-afda-644c0b0ebe39 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:52:04.827244 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 562/1126] 10.222.0.18 () {56 vars in 1177 bytes} [Sat Aug 6 21:52:04 2022] GET /identity/v3/roles/93c058ac49e242fd8517c80c552dea2d => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:52:04.833076 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaab65f2-ea1e-417d-bb5d-a1fb930381f9 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:52:04.929988 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaab65f2-ea1e-417d-bb5d-a1fb930381f9 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:52:04.962109 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381f647b-73a3-443b-b56f-287339e23209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:52:04.962715 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381f647b-73a3-443b-b56f-287339e23209 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:04.962990 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381f647b-73a3-443b-b56f-287339e23209 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:04.963257 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381f647b-73a3-443b-b56f-287339e23209 None admin] PATH_INFO: `/v3/users/e21addffee5e444ca2fc47473f476b88` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:04.964348 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381f647b-73a3-443b-b56f-287339e23209 None admin] RBAC: Authorizing `identity:get_user(user_id=e21addffee5e444ca2fc47473f476b88)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:52:04.966519 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381f647b-73a3-443b-b56f-287339e23209 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:52:04.968069 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 565/1127] 10.222.0.18 () {56 vars in 1177 bytes} [Sat Aug 6 21:52:04 2022] GET /identity/v3/users/e21addffee5e444ca2fc47473f476b88 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:52:04.973959 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7bc1b3-9512-42b7-afda-644c0b0ebe39 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:52:05.025004 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:52:05.070220 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7bc1b3-9512-42b7-afda-644c0b0ebe39 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:52:05.102653 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff13715-37ad-4cf3-810a-dc44b793fe59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:52:05.103235 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff13715-37ad-4cf3-810a-dc44b793fe59 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:05.103513 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff13715-37ad-4cf3-810a-dc44b793fe59 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:05.103779 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff13715-37ad-4cf3-810a-dc44b793fe59 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:05.168963 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff13715-37ad-4cf3-810a-dc44b793fe59 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:52:05.170429 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff13715-37ad-4cf3-810a-dc44b793fe59 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:52:05.219321 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:52:05.266029 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 563/1128] 10.222.0.18 () {58 vars in 1282 bytes} [Sat Aug 6 21:52:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 21:52:05.271873 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381f647b-73a3-443b-b56f-287339e23209 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:52:05.329674 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:52:05.370649 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381f647b-73a3-443b-b56f-287339e23209 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:52:05.403094 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4908e0f1-8190-46b6-bdb0-4fec88894f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:52:05.403710 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4908e0f1-8190-46b6-bdb0-4fec88894f0e None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:05.403986 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4908e0f1-8190-46b6-bdb0-4fec88894f0e None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:05.404252 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4908e0f1-8190-46b6-bdb0-4fec88894f0e None admin] PATH_INFO: `/v3/projects/2d15f49835ea4bcbb21c7d5a53058393` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:05.405457 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4908e0f1-8190-46b6-bdb0-4fec88894f0e None admin] RBAC: Authorizing `identity:get_project(project_id=2d15f49835ea4bcbb21c7d5a53058393)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:52:05.406870 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4908e0f1-8190-46b6-bdb0-4fec88894f0e None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:52:05.408506 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 566/1129] 10.222.0.18 () {56 vars in 1186 bytes} [Sat Aug 6 21:52:05 2022] GET /identity/v3/projects/2d15f49835ea4bcbb21c7d5a53058393 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:52:05.414638 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff13715-37ad-4cf3-810a-dc44b793fe59 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:52:05.512526 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff13715-37ad-4cf3-810a-dc44b793fe59 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:52:05.546122 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c514798-28c0-46c3-a4f2-df780476ce17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:52:05.546713 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c514798-28c0-46c3-a4f2-df780476ce17 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:05.546994 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c514798-28c0-46c3-a4f2-df780476ce17 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:05.547294 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c514798-28c0-46c3-a4f2-df780476ce17 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:05.548008 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c514798-28c0-46c3-a4f2-df780476ce17 None admin] RBAC: Adding query filter params (role.id=93c058ac49e242fd8517c80c552dea2d, scope.project.id=2d15f49835ea4bcbb21c7d5a53058393, user.id=e21addffee5e444ca2fc47473f476b88) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:52:05.548314 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c514798-28c0-46c3-a4f2-df780476ce17 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:52:05.549765 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c514798-28c0-46c3-a4f2-df780476ce17 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:52:05.555413 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 564/1130] 10.222.0.18 () {56 vars in 1374 bytes} [Sat Aug 6 21:52:05 2022] GET /identity/v3/role_assignments?user.id=e21addffee5e444ca2fc47473f476b88&scope.project.id=2d15f49835ea4bcbb21c7d5a53058393&role.id=93c058ac49e242fd8517c80c552dea2d => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:52:07.565543 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4445216c-f0e5-4f5c-83de-7a9384072c85 None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:07.566084 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4445216c-f0e5-4f5c-83de-7a9384072c85 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:07.566084 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4445216c-f0e5-4f5c-83de-7a9384072c85 None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:07.567423 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 567/1131] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:52:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:52:07.572530 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7833902f-5392-42fc-84bc-bcbea588b60a None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:07.572924 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7833902f-5392-42fc-84bc-bcbea588b60a None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:07.573070 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7833902f-5392-42fc-84bc-bcbea588b60a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:07.591447 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-7833902f-5392-42fc-84bc-bcbea588b60a None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:52:07.661527 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-7833902f-5392-42fc-84bc-bcbea588b60a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:52:07.661527 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:52:07.661725 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 565/1132] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:52:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 21:52:07.685601 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4445216c-f0e5-4f5c-83de-7a9384072c85 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:52:07.786027 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4445216c-f0e5-4f5c-83de-7a9384072c85 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:52:07.810741 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:52:07.818162 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e62913b-956f-4eaa-851f-00ba0e92c0ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:52:07.825141 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e62913b-956f-4eaa-851f-00ba0e92c0ea None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:07.825141 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e62913b-956f-4eaa-851f-00ba0e92c0ea None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:07.825141 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e62913b-956f-4eaa-851f-00ba0e92c0ea None admin] PATH_INFO: `/v3/domains/default` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:07.825141 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e62913b-956f-4eaa-851f-00ba0e92c0ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:52:07.825141 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e62913b-956f-4eaa-851f-00ba0e92c0ea None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:52:07.825141 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 568/1133] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:52:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:52:07.830148 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7833902f-5392-42fc-84bc-bcbea588b60a None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:52:07.926683 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7833902f-5392-42fc-84bc-bcbea588b60a None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:52:07.959805 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185c3c24-2e3f-4303-be68-83b30232e514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:52:07.960423 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c3c24-2e3f-4303-be68-83b30232e514 None admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:07.960704 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c3c24-2e3f-4303-be68-83b30232e514 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:07.960974 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c3c24-2e3f-4303-be68-83b30232e514 None admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:07.961617 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185c3c24-2e3f-4303-be68-83b30232e514 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:52:07.963037 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185c3c24-2e3f-4303-be68-83b30232e514 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:52:08.062291 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:52:08.069322 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 566/1134] 10.222.0.18 () {60 vars in 1141 bytes} [Sat Aug 6 21:52:07 2022] POST /identity/v3/projects => generated 311 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 21:52:10.092007 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60d577d-143e-4690-9210-7cdc2169eea3 None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:10.092564 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60d577d-143e-4690-9210-7cdc2169eea3 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:10.092564 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60d577d-143e-4690-9210-7cdc2169eea3 None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:10.093914 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 569/1135] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:52:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:52:10.099183 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd7b55a-ee95-4559-a850-cf279e958b52 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:10.099615 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd7b55a-ee95-4559-a850-cf279e958b52 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:10.099746 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd7b55a-ee95-4559-a850-cf279e958b52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:10.118546 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-7dd7b55a-ee95-4559-a850-cf279e958b52 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:52:10.119711 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-7dd7b55a-ee95-4559-a850-cf279e958b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:52:10.186448 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 567/1136] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:52:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 21:52:10.210897 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60d577d-143e-4690-9210-7cdc2169eea3 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:52:10.309549 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60d577d-143e-4690-9210-7cdc2169eea3 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:52:10.343677 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cbadd2-4895-4dc2-b45d-74e2578b574e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:52:10.344232 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cbadd2-4895-4dc2-b45d-74e2578b574e None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:10.344541 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cbadd2-4895-4dc2-b45d-74e2578b574e None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:10.344822 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cbadd2-4895-4dc2-b45d-74e2578b574e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:10.345948 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cbadd2-4895-4dc2-b45d-74e2578b574e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:52:10.347408 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cbadd2-4895-4dc2-b45d-74e2578b574e None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:52:10.349041 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 570/1137] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:52:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:52:10.354816 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd7b55a-ee95-4559-a850-cf279e958b52 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:52:10.451824 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd7b55a-ee95-4559-a850-cf279e958b52 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:52:10.477603 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:52:10.485966 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e80d40-53b7-4cbe-b82a-b5df3bd1c005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:52:10.486559 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e80d40-53b7-4cbe-b82a-b5df3bd1c005 None admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:10.486838 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e80d40-53b7-4cbe-b82a-b5df3bd1c005 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:10.487110 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e80d40-53b7-4cbe-b82a-b5df3bd1c005 None admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:10.487781 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e80d40-53b7-4cbe-b82a-b5df3bd1c005 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:52:10.489229 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e80d40-53b7-4cbe-b82a-b5df3bd1c005 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:52:10.553088 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 568/1138] 10.222.0.18 () {60 vars in 1132 bytes} [Sat Aug 6 21:52:10 2022] POST /identity/v3/users => generated 286 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 21:52:12.573330 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52db6c8-a57c-456b-8ab5-57c7806b5482 None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:12.573885 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52db6c8-a57c-456b-8ab5-57c7806b5482 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:12.573885 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52db6c8-a57c-456b-8ab5-57c7806b5482 None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:12.575316 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 571/1139] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:52:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:52:12.580334 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c81c87a-239d-44f9-a1db-086ba71993b9 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:12.580782 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c81c87a-239d-44f9-a1db-086ba71993b9 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:12.580913 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c81c87a-239d-44f9-a1db-086ba71993b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:12.599036 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-1c81c87a-239d-44f9-a1db-086ba71993b9 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:52:12.600218 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-1c81c87a-239d-44f9-a1db-086ba71993b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:52:12.664463 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 569/1140] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:52:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 21:52:12.687528 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52db6c8-a57c-456b-8ab5-57c7806b5482 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:52:12.749358 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:52:12.786640 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52db6c8-a57c-456b-8ab5-57c7806b5482 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:52:12.819569 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64772220-3a3e-44a7-9428-2665fd8f1486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:52:12.820185 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64772220-3a3e-44a7-9428-2665fd8f1486 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:12.820487 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64772220-3a3e-44a7-9428-2665fd8f1486 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:12.820760 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64772220-3a3e-44a7-9428-2665fd8f1486 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:12.831000 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64772220-3a3e-44a7-9428-2665fd8f1486 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:52:12.835663 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-64772220-3a3e-44a7-9428-2665fd8f1486 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 21:52:12.835663 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:52:12.835663 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:52:12.835663 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:52:12.835663 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:52:12.835663 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:52:12.835663 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:52:12.835663 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:52:12.835663 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:52:12.835663 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:52:12.835663 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:52:12.835663 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:52:12.835663 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:52:12.835663 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:52:12.835663 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:52:12.835663 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:52:12.836921 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:52:12.836921 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:52:12.836921 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:52:12.836921 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:52:12.836921 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:52:12.836921 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:52:12.836921 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:52:12.836921 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:52:12.836921 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:52:12.836921 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:52:12.836921 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:52:12.836921 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:52:12.836921 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:52:12.836921 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:52:12.836921 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:52:12.836921 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:52:12.838032 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:52:12.838032 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:52:12.838032 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:52:12.838032 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:52:12.838032 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:52:12.838032 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:52:12.838032 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:52:12.838032 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:52:12.838032 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:52:12.838032 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:52:12.838032 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 21:52:12.838032 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:52:12.838032 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 572/1141] 10.222.0.18 () {56 vars in 1096 bytes} [Sat Aug 6 21:52:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:52:12.843400 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c81c87a-239d-44f9-a1db-086ba71993b9 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:52:12.939627 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c81c87a-239d-44f9-a1db-086ba71993b9 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:52:12.953667 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:52:12.973437 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6e4586-98b0-419d-ba1d-f2cb97ea40ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:52:12.974023 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e4586-98b0-419d-ba1d-f2cb97ea40ad None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:12.974302 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e4586-98b0-419d-ba1d-f2cb97ea40ad None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:12.974573 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e4586-98b0-419d-ba1d-f2cb97ea40ad None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:12.975226 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6e4586-98b0-419d-ba1d-f2cb97ea40ad None admin] RBAC: Adding query filter params (name=admin) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:52:12.975526 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6e4586-98b0-419d-ba1d-f2cb97ea40ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:52:12.976919 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6e4586-98b0-419d-ba1d-f2cb97ea40ad None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:52:12.985469 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 570/1142] 10.222.0.18 () {56 vars in 1099 bytes} [Sat Aug 6 21:52:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:52:12.991196 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64772220-3a3e-44a7-9428-2665fd8f1486 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:52:13.086968 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64772220-3a3e-44a7-9428-2665fd8f1486 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:52:13.109415 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:52:13.119495 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a20cdc2-6bfc-466e-aab2-e388252a5907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:52:13.120102 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a20cdc2-6bfc-466e-aab2-e388252a5907 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:13.120375 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a20cdc2-6bfc-466e-aab2-e388252a5907 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:13.120677 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a20cdc2-6bfc-466e-aab2-e388252a5907 None admin] PATH_INFO: `/v3/users/ee8f84c1921a4ee2901c512fa109f87e` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:13.133228 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a20cdc2-6bfc-466e-aab2-e388252a5907 None admin] RBAC: Authorizing `identity:get_user(user_id=ee8f84c1921a4ee2901c512fa109f87e)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:52:13.134701 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a20cdc2-6bfc-466e-aab2-e388252a5907 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:52:13.137008 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 573/1143] 10.222.0.18 () {56 vars in 1177 bytes} [Sat Aug 6 21:52:12 2022] GET /identity/v3/users/ee8f84c1921a4ee2901c512fa109f87e => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:52:13.142459 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6e4586-98b0-419d-ba1d-f2cb97ea40ad None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:52:13.240293 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6e4586-98b0-419d-ba1d-f2cb97ea40ad None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:52:13.273916 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2ba62f-5915-493d-aa07-e855794eedcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:52:13.274507 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2ba62f-5915-493d-aa07-e855794eedcd None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:13.274784 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2ba62f-5915-493d-aa07-e855794eedcd None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:13.275055 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2ba62f-5915-493d-aa07-e855794eedcd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:13.318834 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:52:13.340763 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2ba62f-5915-493d-aa07-e855794eedcd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:52:13.342203 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2ba62f-5915-493d-aa07-e855794eedcd None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:52:13.437937 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 571/1144] 10.222.0.18 () {58 vars in 1282 bytes} [Sat Aug 6 21:52:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 21:52:13.443771 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a20cdc2-6bfc-466e-aab2-e388252a5907 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:52:13.539639 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a20cdc2-6bfc-466e-aab2-e388252a5907 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:52:13.572340 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3af2be-64a0-4221-a1ce-016cd29e216e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:52:13.572984 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3af2be-64a0-4221-a1ce-016cd29e216e None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:13.573264 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3af2be-64a0-4221-a1ce-016cd29e216e None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:13.573538 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3af2be-64a0-4221-a1ce-016cd29e216e None admin] PATH_INFO: `/v3/projects/1a7be97e153d4cf7923614925b9561a5` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:13.574891 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3af2be-64a0-4221-a1ce-016cd29e216e None admin] RBAC: Authorizing `identity:get_project(project_id=1a7be97e153d4cf7923614925b9561a5)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:52:13.576324 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3af2be-64a0-4221-a1ce-016cd29e216e None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:52:13.577955 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 574/1145] 10.222.0.18 () {56 vars in 1186 bytes} [Sat Aug 6 21:52:13 2022] GET /identity/v3/projects/1a7be97e153d4cf7923614925b9561a5 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:52:13.583918 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2ba62f-5915-493d-aa07-e855794eedcd None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:52:13.657401 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:52:13.679386 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2ba62f-5915-493d-aa07-e855794eedcd None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:52:13.712143 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1273075-9918-4671-88d5-26f9e3fc4e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:52:13.712752 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1273075-9918-4671-88d5-26f9e3fc4e56 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:13.713030 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1273075-9918-4671-88d5-26f9e3fc4e56 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:13.713298 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1273075-9918-4671-88d5-26f9e3fc4e56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:13.714005 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1273075-9918-4671-88d5-26f9e3fc4e56 None admin] RBAC: Adding query filter params (role.id=4b8dcf0491fd4072acc69fd894bf77d0, scope.project.id=1a7be97e153d4cf7923614925b9561a5, user.id=ee8f84c1921a4ee2901c512fa109f87e) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:52:13.714308 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1273075-9918-4671-88d5-26f9e3fc4e56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:52:13.715912 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1273075-9918-4671-88d5-26f9e3fc4e56 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:52:13.721137 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 572/1146] 10.222.0.18 () {56 vars in 1374 bytes} [Sat Aug 6 21:52:13 2022] GET /identity/v3/role_assignments?user.id=ee8f84c1921a4ee2901c512fa109f87e&scope.project.id=1a7be97e153d4cf7923614925b9561a5&role.id=4b8dcf0491fd4072acc69fd894bf77d0 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:52:15.688476 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2729dff-bf1e-4d8c-a1de-88458208ae0b None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:15.688982 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2729dff-bf1e-4d8c-a1de-88458208ae0b None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:15.688982 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2729dff-bf1e-4d8c-a1de-88458208ae0b None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:15.690259 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 575/1147] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:52:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:52:15.695305 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f992a37c-fd7f-4720-9d25-b922229cd109 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:15.695737 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f992a37c-fd7f-4720-9d25-b922229cd109 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:15.695854 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f992a37c-fd7f-4720-9d25-b922229cd109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:15.713397 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-f992a37c-fd7f-4720-9d25-b922229cd109 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:52:15.715077 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-f992a37c-fd7f-4720-9d25-b922229cd109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:52:15.779405 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 573/1148] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:52:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 21:52:15.801620 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2729dff-bf1e-4d8c-a1de-88458208ae0b None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:52:15.850464 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:52:15.899181 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2729dff-bf1e-4d8c-a1de-88458208ae0b None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:52:15.932163 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a5bfae-bcec-48d6-beee-966c6e332272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:52:15.932797 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a5bfae-bcec-48d6-beee-966c6e332272 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:15.933070 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a5bfae-bcec-48d6-beee-966c6e332272 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:15.933461 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a5bfae-bcec-48d6-beee-966c6e332272 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:15.943473 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a5bfae-bcec-48d6-beee-966c6e332272 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:52:15.948036 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-25a5bfae-bcec-48d6-beee-966c6e332272 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 21:52:15.948036 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:52:15.948036 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:52:15.948036 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:52:15.948036 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:52:15.948036 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:52:15.948036 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:52:15.948036 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:52:15.948036 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:52:15.948036 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:52:15.948036 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:52:15.948036 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:52:15.948036 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:52:15.948036 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:52:15.948036 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:52:15.948036 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:52:15.949672 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:52:15.949672 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:52:15.949672 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:52:15.949672 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:52:15.949672 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:52:15.949672 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:52:15.949672 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:52:15.949672 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:52:15.949672 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:52:15.949672 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:52:15.949672 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:52:15.949672 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:52:15.949672 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:52:15.949672 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:52:15.949672 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:52:15.949672 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:52:15.950668 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:52:15.950668 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:52:15.950668 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:52:15.950668 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:52:15.950668 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:52:15.950668 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:52:15.950668 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:52:15.950668 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:52:15.950668 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:52:15.950668 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:52:15.950668 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 21:52:15.950668 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:52:15.950668 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 576/1149] 10.222.0.18 () {56 vars in 1096 bytes} [Sat Aug 6 21:52:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:52:15.955654 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f992a37c-fd7f-4720-9d25-b922229cd109 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:52:16.022857 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:52:16.052127 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f992a37c-fd7f-4720-9d25-b922229cd109 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:52:16.085173 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4b2040-dbf1-4263-a65b-e14b1b5ed967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:52:16.085764 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4b2040-dbf1-4263-a65b-e14b1b5ed967 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:16.086040 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4b2040-dbf1-4263-a65b-e14b1b5ed967 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:16.086309 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4b2040-dbf1-4263-a65b-e14b1b5ed967 None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:16.086963 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4b2040-dbf1-4263-a65b-e14b1b5ed967 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:52:16.087263 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4b2040-dbf1-4263-a65b-e14b1b5ed967 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:52:16.088644 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4b2040-dbf1-4263-a65b-e14b1b5ed967 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:52:16.097288 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 574/1150] 10.222.0.18 () {56 vars in 1099 bytes} [Sat Aug 6 21:52:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:52:16.103005 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a5bfae-bcec-48d6-beee-966c6e332272 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:52:16.206524 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a5bfae-bcec-48d6-beee-966c6e332272 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:52:16.216964 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:52:16.239302 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094b0370-e6f4-49fa-9675-3dff6657f2a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:52:16.239913 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094b0370-e6f4-49fa-9675-3dff6657f2a4 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:16.240189 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094b0370-e6f4-49fa-9675-3dff6657f2a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:16.240488 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094b0370-e6f4-49fa-9675-3dff6657f2a4 None admin] PATH_INFO: `/v3/users/ee8f84c1921a4ee2901c512fa109f87e` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:16.241600 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094b0370-e6f4-49fa-9675-3dff6657f2a4 None admin] RBAC: Authorizing `identity:get_user(user_id=ee8f84c1921a4ee2901c512fa109f87e)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:52:16.243070 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094b0370-e6f4-49fa-9675-3dff6657f2a4 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:52:16.244873 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 577/1151] 10.222.0.18 () {56 vars in 1177 bytes} [Sat Aug 6 21:52:16 2022] GET /identity/v3/users/ee8f84c1921a4ee2901c512fa109f87e => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:52:16.250559 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4b2040-dbf1-4263-a65b-e14b1b5ed967 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:52:16.349783 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4b2040-dbf1-4263-a65b-e14b1b5ed967 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:52:16.378544 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:52:16.383260 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada9c7d7-b235-45d7-ae5c-965cb16a9c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:52:16.383888 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada9c7d7-b235-45d7-ae5c-965cb16a9c47 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:16.384166 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada9c7d7-b235-45d7-ae5c-965cb16a9c47 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:16.384468 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada9c7d7-b235-45d7-ae5c-965cb16a9c47 None admin] PATH_INFO: `/v3/projects/1a7be97e153d4cf7923614925b9561a5` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:16.385570 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada9c7d7-b235-45d7-ae5c-965cb16a9c47 None admin] RBAC: Authorizing `identity:get_project(project_id=1a7be97e153d4cf7923614925b9561a5)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:52:16.387030 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada9c7d7-b235-45d7-ae5c-965cb16a9c47 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:52:16.388663 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 575/1152] 10.222.0.18 () {56 vars in 1186 bytes} [Sat Aug 6 21:52:16 2022] GET /identity/v3/projects/1a7be97e153d4cf7923614925b9561a5 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:52:16.394605 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094b0370-e6f4-49fa-9675-3dff6657f2a4 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:52:16.492613 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094b0370-e6f4-49fa-9675-3dff6657f2a4 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:52:16.524097 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cc5580-ef9b-41a3-8281-7ac6c2b40574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:52:16.524761 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc5580-ef9b-41a3-8281-7ac6c2b40574 None admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:16.525038 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc5580-ef9b-41a3-8281-7ac6c2b40574 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:16.525306 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc5580-ef9b-41a3-8281-7ac6c2b40574 None admin] PATH_INFO: `/v3/projects/1a7be97e153d4cf7923614925b9561a5/users/ee8f84c1921a4ee2901c512fa109f87e/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:16.527696 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cc5580-ef9b-41a3-8281-7ac6c2b40574 None admin] RBAC: Authorizing `identity:create_grant(project_id=1a7be97e153d4cf7923614925b9561a5, user_id=ee8f84c1921a4ee2901c512fa109f87e, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:52:16.529265 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cc5580-ef9b-41a3-8281-7ac6c2b40574 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:52:16.604281 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 578/1153] 10.222.0.18 () {58 vars in 1439 bytes} [Sat Aug 6 21:52:16 2022] PUT /identity/v3/projects/1a7be97e153d4cf7923614925b9561a5/users/ee8f84c1921a4ee2901c512fa109f87e/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 21:52:18.564101 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da17c72-55bc-4a01-b11c-99204c692f1c None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:18.564652 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da17c72-55bc-4a01-b11c-99204c692f1c None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:18.564771 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da17c72-55bc-4a01-b11c-99204c692f1c None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:18.566008 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 576/1154] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:52:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:52:18.571255 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d332965d-3152-4c1f-84ca-89959ac17b39 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:18.571639 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d332965d-3152-4c1f-84ca-89959ac17b39 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:18.571828 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d332965d-3152-4c1f-84ca-89959ac17b39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:18.589320 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-d332965d-3152-4c1f-84ca-89959ac17b39 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:52:18.591000 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-d332965d-3152-4c1f-84ca-89959ac17b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:52:18.619810 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:52:18.655455 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 579/1155] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:52:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 21:52:18.678238 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da17c72-55bc-4a01-b11c-99204c692f1c None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:52:18.778514 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da17c72-55bc-4a01-b11c-99204c692f1c None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:52:18.811234 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93768c6-1109-4b7e-b976-be31e79f1bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:52:18.811845 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93768c6-1109-4b7e-b976-be31e79f1bb1 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:18.812121 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93768c6-1109-4b7e-b976-be31e79f1bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:18.812390 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93768c6-1109-4b7e-b976-be31e79f1bb1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:18.822627 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93768c6-1109-4b7e-b976-be31e79f1bb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:52:18.827908 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-f93768c6-1109-4b7e-b976-be31e79f1bb1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 21:52:18.827908 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:52:18.827908 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:52:18.827908 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:52:18.827908 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:52:18.827908 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:52:18.827908 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:52:18.827908 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:52:18.827908 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:52:18.827908 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:52:18.827908 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:52:18.827908 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:52:18.827908 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:52:18.827908 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:52:18.827908 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:52:18.827908 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:52:18.829130 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:52:18.829130 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:52:18.829130 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:52:18.829130 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:52:18.829130 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:52:18.829130 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:52:18.829130 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:52:18.829130 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:52:18.829130 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:52:18.829130 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:52:18.829130 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:52:18.829130 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:52:18.829130 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:52:18.829130 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:52:18.829130 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:52:18.829130 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:52:18.830187 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:52:18.830187 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:52:18.830187 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:52:18.830187 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:52:18.830187 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:52:18.830187 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:52:18.830187 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:52:18.830187 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:52:18.830187 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:52:18.830187 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:52:18.830187 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 21:52:18.830187 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:52:18.830187 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 577/1156] 10.222.0.18 () {56 vars in 1096 bytes} [Sat Aug 6 21:52:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:52:18.834879 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d332965d-3152-4c1f-84ca-89959ac17b39 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:52:18.931711 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d332965d-3152-4c1f-84ca-89959ac17b39 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:52:18.965285 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50788fa3-abc6-4e1f-93de-054f98056abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:52:18.965879 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50788fa3-abc6-4e1f-93de-054f98056abc None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:18.966155 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50788fa3-abc6-4e1f-93de-054f98056abc None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:18.966418 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50788fa3-abc6-4e1f-93de-054f98056abc None admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:18.967069 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50788fa3-abc6-4e1f-93de-054f98056abc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:52:18.967365 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50788fa3-abc6-4e1f-93de-054f98056abc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:52:18.968739 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50788fa3-abc6-4e1f-93de-054f98056abc None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:52:18.976459 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 580/1157] 10.222.0.18 () {56 vars in 1099 bytes} [Sat Aug 6 21:52:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:52:18.983276 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93768c6-1109-4b7e-b976-be31e79f1bb1 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:52:19.040353 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:52:19.079521 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93768c6-1109-4b7e-b976-be31e79f1bb1 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:52:19.112123 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e986a60-b5e7-4008-9ee1-2ee8ca920886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:52:19.112755 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e986a60-b5e7-4008-9ee1-2ee8ca920886 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:19.113029 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e986a60-b5e7-4008-9ee1-2ee8ca920886 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:19.113295 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e986a60-b5e7-4008-9ee1-2ee8ca920886 None admin] PATH_INFO: `/v3/users/ee8f84c1921a4ee2901c512fa109f87e` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:19.114389 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e986a60-b5e7-4008-9ee1-2ee8ca920886 None admin] RBAC: Authorizing `identity:get_user(user_id=ee8f84c1921a4ee2901c512fa109f87e)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:52:19.115803 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e986a60-b5e7-4008-9ee1-2ee8ca920886 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:52:19.117369 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 578/1158] 10.222.0.18 () {56 vars in 1177 bytes} [Sat Aug 6 21:52:18 2022] GET /identity/v3/users/ee8f84c1921a4ee2901c512fa109f87e => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:52:19.123878 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50788fa3-abc6-4e1f-93de-054f98056abc None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:52:19.153816 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:52:19.221145 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50788fa3-abc6-4e1f-93de-054f98056abc None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:52:19.255190 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46399d24-7ea8-4fab-864e-7548663efe00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:52:19.255775 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46399d24-7ea8-4fab-864e-7548663efe00 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:19.256053 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46399d24-7ea8-4fab-864e-7548663efe00 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:19.256323 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46399d24-7ea8-4fab-864e-7548663efe00 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:19.320954 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46399d24-7ea8-4fab-864e-7548663efe00 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:52:19.322374 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46399d24-7ea8-4fab-864e-7548663efe00 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:52:19.351320 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:52:19.419416 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 581/1159] 10.222.0.18 () {58 vars in 1282 bytes} [Sat Aug 6 21:52:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 21:52:19.425318 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e986a60-b5e7-4008-9ee1-2ee8ca920886 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:52:19.523575 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e986a60-b5e7-4008-9ee1-2ee8ca920886 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:52:19.549648 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:52:19.559454 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d092045-97c2-4be6-b7fc-87b7172ccfd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:52:19.560062 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d092045-97c2-4be6-b7fc-87b7172ccfd0 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:19.560334 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d092045-97c2-4be6-b7fc-87b7172ccfd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:19.562696 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d092045-97c2-4be6-b7fc-87b7172ccfd0 None admin] PATH_INFO: `/v3/projects/1a7be97e153d4cf7923614925b9561a5` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:19.562696 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d092045-97c2-4be6-b7fc-87b7172ccfd0 None admin] RBAC: Authorizing `identity:get_project(project_id=1a7be97e153d4cf7923614925b9561a5)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:52:19.563123 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d092045-97c2-4be6-b7fc-87b7172ccfd0 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:52:19.564746 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 579/1160] 10.222.0.18 () {56 vars in 1186 bytes} [Sat Aug 6 21:52:19 2022] GET /identity/v3/projects/1a7be97e153d4cf7923614925b9561a5 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:52:19.570815 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46399d24-7ea8-4fab-864e-7548663efe00 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:52:19.667601 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46399d24-7ea8-4fab-864e-7548663efe00 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:52:19.694706 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:52:19.700450 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e6a477-bfb5-4e34-bb15-337a55a23fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:52:19.707046 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e6a477-bfb5-4e34-bb15-337a55a23fb7 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:19.707046 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e6a477-bfb5-4e34-bb15-337a55a23fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:19.707046 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e6a477-bfb5-4e34-bb15-337a55a23fb7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:19.707046 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e6a477-bfb5-4e34-bb15-337a55a23fb7 None admin] RBAC: Adding query filter params (role.id=4b8dcf0491fd4072acc69fd894bf77d0, scope.project.id=1a7be97e153d4cf7923614925b9561a5, user.id=ee8f84c1921a4ee2901c512fa109f87e) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:52:19.707046 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e6a477-bfb5-4e34-bb15-337a55a23fb7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:52:19.707046 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e6a477-bfb5-4e34-bb15-337a55a23fb7 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:52:19.709731 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 582/1161] 10.222.0.18 () {56 vars in 1374 bytes} [Sat Aug 6 21:52:19 2022] GET /identity/v3/role_assignments?user.id=ee8f84c1921a4ee2901c512fa109f87e&scope.project.id=1a7be97e153d4cf7923614925b9561a5&role.id=4b8dcf0491fd4072acc69fd894bf77d0 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:52:21.730395 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a02d0c-af7f-48ab-9181-848eb2bdb13d None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:21.730924 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a02d0c-af7f-48ab-9181-848eb2bdb13d None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:21.730924 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a02d0c-af7f-48ab-9181-848eb2bdb13d None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:21.732574 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 580/1162] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:52:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:52:21.738149 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e5e7b9-c756-4313-ae18-a9824d4b8473 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:21.738541 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e5e7b9-c756-4313-ae18-a9824d4b8473 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:21.738689 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e5e7b9-c756-4313-ae18-a9824d4b8473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:21.756730 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-b9e5e7b9-c756-4313-ae18-a9824d4b8473 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:52:21.757887 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-b9e5e7b9-c756-4313-ae18-a9824d4b8473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:52:21.821728 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 583/1163] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:52:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 21:52:21.844129 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a02d0c-af7f-48ab-9181-848eb2bdb13d None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:52:21.944077 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a02d0c-af7f-48ab-9181-848eb2bdb13d None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:52:21.976210 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db7aa55-8a8c-4206-b061-5546acea20a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:52:21.976835 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db7aa55-8a8c-4206-b061-5546acea20a6 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:21.977111 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db7aa55-8a8c-4206-b061-5546acea20a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:21.977377 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db7aa55-8a8c-4206-b061-5546acea20a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:22.035838 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:52:22.042035 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db7aa55-8a8c-4206-b061-5546acea20a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:52:22.044258 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db7aa55-8a8c-4206-b061-5546acea20a6 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:52:22.137908 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 581/1164] 10.222.0.18 () {58 vars in 1282 bytes} [Sat Aug 6 21:52:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 21:52:22.143874 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e5e7b9-c756-4313-ae18-a9824d4b8473 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:52:22.238494 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e5e7b9-c756-4313-ae18-a9824d4b8473 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:52:22.261042 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:52:22.271167 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2effbc-296f-42d3-950b-87f814ba6a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:52:22.271778 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2effbc-296f-42d3-950b-87f814ba6a65 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:22.273165 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2effbc-296f-42d3-950b-87f814ba6a65 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:22.273165 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2effbc-296f-42d3-950b-87f814ba6a65 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:22.278703 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2effbc-296f-42d3-950b-87f814ba6a65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:52:22.280116 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2effbc-296f-42d3-950b-87f814ba6a65 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:52:22.290910 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-8e2effbc-296f-42d3-950b-87f814ba6a65 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 06 21:52:22.290910 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:52:22.290910 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 06 21:52:22.290910 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 06 21:52:22.290910 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 21:52:22.290910 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 21:52:22.290910 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 21:52:22.290910 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 21:52:22.290910 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 06 21:52:22.290910 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:52:22.290910 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 21:52:22.290910 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:52:22.290910 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:52:22.290910 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:52:22.290910 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:52:22.290910 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:52:22.290910 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:52:22.292165 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:52:22.292165 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:52:22.292165 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:52:22.292165 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:52:22.292165 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:52:22.292165 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:52:22.292165 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:52:22.292165 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:52:22.292165 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:52:22.292165 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:52:22.292165 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:52:22.292165 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:52:22.292165 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:52:22.292165 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:52:22.292165 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:52:22.292165 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:52:22.294051 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:52:22.294051 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:52:22.294051 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:52:22.294051 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:52:22.294051 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:52:22.294051 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:52:22.294051 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:52:22.294051 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:52:22.294051 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:52:22.294051 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:52:22.294051 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 06 21:52:22.294051 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:52:22.294051 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 06 21:52:22.294051 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:52:22.294051 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 584/1165] 10.222.0.18 () {56 vars in 1117 bytes} [Sat Aug 6 21:52:22 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:52:22.298354 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db7aa55-8a8c-4206-b061-5546acea20a6 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:52:22.390017 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:52:22.395832 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db7aa55-8a8c-4206-b061-5546acea20a6 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:52:22.428519 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2620f48f-ed9b-47a0-9b34-0bef8d8a1611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:52:22.429110 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2620f48f-ed9b-47a0-9b34-0bef8d8a1611 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:22.429386 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2620f48f-ed9b-47a0-9b34-0bef8d8a1611 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:22.429655 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2620f48f-ed9b-47a0-9b34-0bef8d8a1611 None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:22.430284 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2620f48f-ed9b-47a0-9b34-0bef8d8a1611 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:52:22.430582 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2620f48f-ed9b-47a0-9b34-0bef8d8a1611 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:52:22.431963 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2620f48f-ed9b-47a0-9b34-0bef8d8a1611 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:52:22.439358 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 582/1166] 10.222.0.18 () {56 vars in 1115 bytes} [Sat Aug 6 21:52:22 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:52:24.412375 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfa9254-5906-4cc9-b280-ff695a2f8da7 None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:24.412979 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfa9254-5906-4cc9-b280-ff695a2f8da7 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:24.412979 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfa9254-5906-4cc9-b280-ff695a2f8da7 None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:24.414265 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 585/1167] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:52:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:52:24.423744 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ede6fa8-0560-42cf-840a-62e09c84bb69 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:24.424144 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ede6fa8-0560-42cf-840a-62e09c84bb69 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:24.424284 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ede6fa8-0560-42cf-840a-62e09c84bb69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:24.441777 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-0ede6fa8-0560-42cf-840a-62e09c84bb69 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:52:24.442957 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-0ede6fa8-0560-42cf-840a-62e09c84bb69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:52:24.508340 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 583/1168] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:52:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 21:52:24.530497 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfa9254-5906-4cc9-b280-ff695a2f8da7 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:52:24.628885 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfa9254-5906-4cc9-b280-ff695a2f8da7 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:52:24.661320 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11037738-e220-4155-8b86-1def8cfd19f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:52:24.661908 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11037738-e220-4155-8b86-1def8cfd19f0 None admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:24.662186 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11037738-e220-4155-8b86-1def8cfd19f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:24.662458 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11037738-e220-4155-8b86-1def8cfd19f0 None admin] PATH_INFO: `/v3/domains` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:24.663115 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11037738-e220-4155-8b86-1def8cfd19f0 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:52:24.664503 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11037738-e220-4155-8b86-1def8cfd19f0 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:52:24.748190 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 586/1169] 10.222.0.18 () {60 vars in 1138 bytes} [Sat Aug 6 21:52:24 2022] POST /identity/v3/domains => generated 267 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 21:52:26.756896 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7838e41-d6bf-4c1b-9736-045465393007 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:26.757409 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7838e41-d6bf-4c1b-9736-045465393007 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:26.757409 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7838e41-d6bf-4c1b-9736-045465393007 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:26.758760 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 584/1170] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:52:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:52:26.764779 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2834e0fe-75ef-4ff5-a268-0560224c630a None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:26.765163 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2834e0fe-75ef-4ff5-a268-0560224c630a None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:26.765333 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2834e0fe-75ef-4ff5-a268-0560224c630a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:26.784420 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-2834e0fe-75ef-4ff5-a268-0560224c630a None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:52:26.785620 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-2834e0fe-75ef-4ff5-a268-0560224c630a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:52:26.825734 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:52:26.850701 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 587/1171] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:52:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 21:52:26.875275 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7838e41-d6bf-4c1b-9736-045465393007 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:52:26.967304 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:52:26.977465 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7838e41-d6bf-4c1b-9736-045465393007 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:52:27.010906 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d640c0d-a00b-4aa1-ba05-a94b79f68ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:52:27.011617 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d640c0d-a00b-4aa1-ba05-a94b79f68ab2 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:27.011891 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d640c0d-a00b-4aa1-ba05-a94b79f68ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:27.012158 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d640c0d-a00b-4aa1-ba05-a94b79f68ab2 None admin] PATH_INFO: `/v3/domains/78a3ee7fce144eceb6ded3449e1208e0` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:27.022044 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d640c0d-a00b-4aa1-ba05-a94b79f68ab2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=78a3ee7fce144eceb6ded3449e1208e0)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:52:27.023470 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d640c0d-a00b-4aa1-ba05-a94b79f68ab2 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:52:27.025042 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 585/1172] 10.222.0.18 () {56 vars in 1183 bytes} [Sat Aug 6 21:52:26 2022] GET /identity/v3/domains/78a3ee7fce144eceb6ded3449e1208e0 => generated 267 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:52:27.031680 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2834e0fe-75ef-4ff5-a268-0560224c630a None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:52:27.126776 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2834e0fe-75ef-4ff5-a268-0560224c630a None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:52:27.159242 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7bc371-066e-4a56-bfba-1ebd472dabc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:52:27.159824 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7bc371-066e-4a56-bfba-1ebd472dabc5 None admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:27.160096 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7bc371-066e-4a56-bfba-1ebd472dabc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:27.160359 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7bc371-066e-4a56-bfba-1ebd472dabc5 None admin] PATH_INFO: `/v3/projects` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:27.161019 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7bc371-066e-4a56-bfba-1ebd472dabc5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:52:27.162413 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7bc371-066e-4a56-bfba-1ebd472dabc5 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:52:27.250383 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 588/1173] 10.222.0.18 () {60 vars in 1141 bytes} [Sat Aug 6 21:52:27 2022] POST /identity/v3/projects => generated 361 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 21:52:29.254451 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e682704-999d-4ba6-9881-94cd728aa95c None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:29.255041 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e682704-999d-4ba6-9881-94cd728aa95c None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:29.255041 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e682704-999d-4ba6-9881-94cd728aa95c None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:29.256313 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 586/1174] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:52:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:52:29.261352 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a63ddae-b4e4-4323-87dc-6de992b8c338 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:29.261791 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a63ddae-b4e4-4323-87dc-6de992b8c338 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:29.261898 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a63ddae-b4e4-4323-87dc-6de992b8c338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:29.280075 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-1a63ddae-b4e4-4323-87dc-6de992b8c338 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:52:29.281308 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-1a63ddae-b4e4-4323-87dc-6de992b8c338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:52:29.301292 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:52:29.345910 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 589/1175] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:52:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 21:52:29.370882 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e682704-999d-4ba6-9881-94cd728aa95c None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:52:29.470688 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e682704-999d-4ba6-9881-94cd728aa95c None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:52:29.502573 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3929a10-371a-4652-825b-7093e06194a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:52:29.503177 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3929a10-371a-4652-825b-7093e06194a1 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:29.503455 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3929a10-371a-4652-825b-7093e06194a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:29.503721 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3929a10-371a-4652-825b-7093e06194a1 None admin] PATH_INFO: `/v3/domains/78a3ee7fce144eceb6ded3449e1208e0` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:29.504880 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3929a10-371a-4652-825b-7093e06194a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=78a3ee7fce144eceb6ded3449e1208e0)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:52:29.506240 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3929a10-371a-4652-825b-7093e06194a1 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:52:29.507751 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 587/1176] 10.222.0.18 () {56 vars in 1183 bytes} [Sat Aug 6 21:52:29 2022] GET /identity/v3/domains/78a3ee7fce144eceb6ded3449e1208e0 => generated 267 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:52:29.513992 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a63ddae-b4e4-4323-87dc-6de992b8c338 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:52:29.609673 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a63ddae-b4e4-4323-87dc-6de992b8c338 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:52:29.642248 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dffbdb8-84b6-456d-98f4-f14f28e2a3d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:52:29.642834 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dffbdb8-84b6-456d-98f4-f14f28e2a3d0 None admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:29.643110 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dffbdb8-84b6-456d-98f4-f14f28e2a3d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:29.643379 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dffbdb8-84b6-456d-98f4-f14f28e2a3d0 None admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:29.644021 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dffbdb8-84b6-456d-98f4-f14f28e2a3d0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:52:29.645457 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dffbdb8-84b6-456d-98f4-f14f28e2a3d0 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:52:29.717924 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 590/1177] 10.222.0.18 () {60 vars in 1132 bytes} [Sat Aug 6 21:52:29 2022] POST /identity/v3/users => generated 311 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 21:52:31.758554 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c394d9-1bc7-442d-896a-4486be66e7d9 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:31.759100 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c394d9-1bc7-442d-896a-4486be66e7d9 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:31.759100 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c394d9-1bc7-442d-896a-4486be66e7d9 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:31.760450 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 588/1178] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:52:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:52:31.766165 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd21dd23-21a1-49df-a01d-16cdfbfb1801 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:31.766578 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd21dd23-21a1-49df-a01d-16cdfbfb1801 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:31.766712 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd21dd23-21a1-49df-a01d-16cdfbfb1801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:31.784951 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-dd21dd23-21a1-49df-a01d-16cdfbfb1801 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:52:31.786118 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-dd21dd23-21a1-49df-a01d-16cdfbfb1801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:52:31.793412 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:52:31.850892 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 591/1179] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:52:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 21:52:31.873830 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c394d9-1bc7-442d-896a-4486be66e7d9 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:52:31.893109 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:52:31.973444 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c394d9-1bc7-442d-896a-4486be66e7d9 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:52:32.005129 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aafe8be-0f67-48a5-8e12-91c4aaf31a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:52:32.005733 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aafe8be-0f67-48a5-8e12-91c4aaf31a7d None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:32.006007 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aafe8be-0f67-48a5-8e12-91c4aaf31a7d None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:32.006274 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aafe8be-0f67-48a5-8e12-91c4aaf31a7d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:32.017026 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aafe8be-0f67-48a5-8e12-91c4aaf31a7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:52:32.021614 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-8aafe8be-0f67-48a5-8e12-91c4aaf31a7d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 21:52:32.021614 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:52:32.021614 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:52:32.021614 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:52:32.021614 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:52:32.021614 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:52:32.021614 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:52:32.021614 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:52:32.021614 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:52:32.021614 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:52:32.021614 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:52:32.021614 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:52:32.021614 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:52:32.021614 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:52:32.021614 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:52:32.021614 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:52:32.022782 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:52:32.022782 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:52:32.022782 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:52:32.022782 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:52:32.022782 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:52:32.022782 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:52:32.022782 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:52:32.022782 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:52:32.022782 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:52:32.022782 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:52:32.022782 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:52:32.022782 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:52:32.022782 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:52:32.022782 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:52:32.022782 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:52:32.022782 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:52:32.023887 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:52:32.023887 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:52:32.023887 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:52:32.023887 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:52:32.023887 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:52:32.023887 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:52:32.023887 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:52:32.023887 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:52:32.023887 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:52:32.023887 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:52:32.023887 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 21:52:32.023887 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:52:32.023887 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 589/1180] 10.222.0.18 () {56 vars in 1096 bytes} [Sat Aug 6 21:52:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:52:32.029362 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd21dd23-21a1-49df-a01d-16cdfbfb1801 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:52:32.125043 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd21dd23-21a1-49df-a01d-16cdfbfb1801 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:52:32.157690 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6244326d-d554-4170-9edd-28c742fb0139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:52:32.158285 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6244326d-d554-4170-9edd-28c742fb0139 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:32.158565 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6244326d-d554-4170-9edd-28c742fb0139 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:32.158836 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6244326d-d554-4170-9edd-28c742fb0139 None admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:32.159495 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6244326d-d554-4170-9edd-28c742fb0139 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:52:32.159795 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6244326d-d554-4170-9edd-28c742fb0139 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:52:32.161177 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6244326d-d554-4170-9edd-28c742fb0139 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:52:32.169221 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 592/1181] 10.222.0.18 () {56 vars in 1099 bytes} [Sat Aug 6 21:52:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:52:32.174972 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aafe8be-0f67-48a5-8e12-91c4aaf31a7d None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:52:32.251680 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:52:32.272479 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aafe8be-0f67-48a5-8e12-91c4aaf31a7d None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:52:32.304957 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc2a346-70db-46e5-bf98-70e9ae9ad0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:52:32.305566 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc2a346-70db-46e5-bf98-70e9ae9ad0f4 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:32.305845 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc2a346-70db-46e5-bf98-70e9ae9ad0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:32.306114 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc2a346-70db-46e5-bf98-70e9ae9ad0f4 None admin] PATH_INFO: `/v3/users/2f72c036028949e684d050734fcdc812` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:32.319218 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc2a346-70db-46e5-bf98-70e9ae9ad0f4 None admin] RBAC: Authorizing `identity:get_user(user_id=2f72c036028949e684d050734fcdc812)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:52:32.320682 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc2a346-70db-46e5-bf98-70e9ae9ad0f4 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:52:32.322250 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 590/1182] 10.222.0.18 () {56 vars in 1177 bytes} [Sat Aug 6 21:52:32 2022] GET /identity/v3/users/2f72c036028949e684d050734fcdc812 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:52:32.328692 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6244326d-d554-4170-9edd-28c742fb0139 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:52:32.332368 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:52:32.425869 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6244326d-d554-4170-9edd-28c742fb0139 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:52:32.458559 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6a2f9c-0dc9-4219-9eaf-f983791446b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:52:32.459153 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6a2f9c-0dc9-4219-9eaf-f983791446b1 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:32.459428 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6a2f9c-0dc9-4219-9eaf-f983791446b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:32.459695 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6a2f9c-0dc9-4219-9eaf-f983791446b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:32.523373 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6a2f9c-0dc9-4219-9eaf-f983791446b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:52:32.524754 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6a2f9c-0dc9-4219-9eaf-f983791446b1 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:52:32.527306 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:52:32.619908 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 593/1183] 10.222.0.18 () {58 vars in 1282 bytes} [Sat Aug 6 21:52:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 21:52:32.625756 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc2a346-70db-46e5-bf98-70e9ae9ad0f4 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:52:32.722011 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc2a346-70db-46e5-bf98-70e9ae9ad0f4 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:52:32.753884 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4c1009-678b-415b-a7a7-33e486316b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:52:32.754489 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4c1009-678b-415b-a7a7-33e486316b23 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:32.754763 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4c1009-678b-415b-a7a7-33e486316b23 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:32.755029 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4c1009-678b-415b-a7a7-33e486316b23 None admin] PATH_INFO: `/v3/projects/45f3dac2007b4895a1040d2ddad4be03` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:32.756113 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4c1009-678b-415b-a7a7-33e486316b23 None admin] RBAC: Authorizing `identity:get_project(project_id=45f3dac2007b4895a1040d2ddad4be03)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:52:32.757556 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4c1009-678b-415b-a7a7-33e486316b23 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:52:32.759154 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 591/1184] 10.222.0.18 () {56 vars in 1186 bytes} [Sat Aug 6 21:52:32 2022] GET /identity/v3/projects/45f3dac2007b4895a1040d2ddad4be03 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:52:32.765608 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6a2f9c-0dc9-4219-9eaf-f983791446b1 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:52:32.861148 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6a2f9c-0dc9-4219-9eaf-f983791446b1 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:52:32.863320 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:52:32.893510 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1d2651-df9d-4e5e-8616-e5e9ed054946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:52:32.894106 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d2651-df9d-4e5e-8616-e5e9ed054946 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:32.894386 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d2651-df9d-4e5e-8616-e5e9ed054946 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:32.894685 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d2651-df9d-4e5e-8616-e5e9ed054946 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:32.895396 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1d2651-df9d-4e5e-8616-e5e9ed054946 None admin] RBAC: Adding query filter params (role.id=4b8dcf0491fd4072acc69fd894bf77d0, scope.project.id=45f3dac2007b4895a1040d2ddad4be03, user.id=2f72c036028949e684d050734fcdc812) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:52:32.895702 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1d2651-df9d-4e5e-8616-e5e9ed054946 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:52:32.897140 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1d2651-df9d-4e5e-8616-e5e9ed054946 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:52:32.903631 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 594/1185] 10.222.0.18 () {56 vars in 1374 bytes} [Sat Aug 6 21:52:32 2022] GET /identity/v3/role_assignments?user.id=2f72c036028949e684d050734fcdc812&scope.project.id=45f3dac2007b4895a1040d2ddad4be03&role.id=4b8dcf0491fd4072acc69fd894bf77d0 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:52:34.860110 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb423bca-9bec-4265-bd2e-09107de88892 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:34.860668 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb423bca-9bec-4265-bd2e-09107de88892 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:34.860780 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb423bca-9bec-4265-bd2e-09107de88892 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:34.862014 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 592/1186] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:52:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:52:34.867040 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75ec5d0-4ce1-47e2-a046-c050e5cb0c3d None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:34.867453 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75ec5d0-4ce1-47e2-a046-c050e5cb0c3d None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:34.867590 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75ec5d0-4ce1-47e2-a046-c050e5cb0c3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:34.886032 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-b75ec5d0-4ce1-47e2-a046-c050e5cb0c3d None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:52:34.887202 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-b75ec5d0-4ce1-47e2-a046-c050e5cb0c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:52:34.951242 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 595/1187] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:52:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 21:52:34.973024 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb423bca-9bec-4265-bd2e-09107de88892 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:52:34.976850 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:52:35.072321 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb423bca-9bec-4265-bd2e-09107de88892 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:52:35.105408 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9f0c45-f595-4d95-bb75-3be0b7e990ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:52:35.106018 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9f0c45-f595-4d95-bb75-3be0b7e990ed None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:35.106295 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9f0c45-f595-4d95-bb75-3be0b7e990ed None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:35.106563 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9f0c45-f595-4d95-bb75-3be0b7e990ed None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:35.116579 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9f0c45-f595-4d95-bb75-3be0b7e990ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:52:35.121796 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-ed9f0c45-f595-4d95-bb75-3be0b7e990ed None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 21:52:35.121796 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:52:35.121796 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:52:35.121796 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:52:35.121796 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:52:35.121796 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:52:35.121796 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:52:35.121796 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:52:35.121796 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:52:35.121796 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:52:35.121796 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:52:35.121796 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:52:35.121796 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:52:35.121796 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:52:35.121796 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:52:35.121796 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:52:35.122938 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:52:35.122938 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:52:35.122938 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:52:35.122938 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:52:35.122938 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:52:35.122938 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:52:35.122938 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:52:35.122938 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:52:35.122938 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:52:35.122938 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:52:35.122938 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:52:35.122938 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:52:35.122938 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:52:35.122938 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:52:35.122938 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:52:35.122938 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:52:35.123990 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:52:35.123990 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:52:35.123990 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:52:35.123990 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:52:35.123990 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:52:35.123990 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:52:35.123990 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:52:35.123990 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:52:35.123990 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:52:35.123990 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:52:35.123990 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 21:52:35.123990 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:52:35.123990 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 593/1188] 10.222.0.18 () {56 vars in 1096 bytes} [Sat Aug 6 21:52:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:52:35.128891 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75ec5d0-4ce1-47e2-a046-c050e5cb0c3d None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:52:35.221822 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:52:35.225517 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75ec5d0-4ce1-47e2-a046-c050e5cb0c3d None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:52:35.259032 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7ee603-219f-4d6c-99c9-6c12e2760af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:52:35.259628 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7ee603-219f-4d6c-99c9-6c12e2760af5 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:35.259909 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7ee603-219f-4d6c-99c9-6c12e2760af5 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:35.260184 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7ee603-219f-4d6c-99c9-6c12e2760af5 None admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:35.261063 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7ee603-219f-4d6c-99c9-6c12e2760af5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:52:35.261366 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7ee603-219f-4d6c-99c9-6c12e2760af5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:52:35.262732 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7ee603-219f-4d6c-99c9-6c12e2760af5 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:52:35.270521 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 596/1189] 10.222.0.18 () {56 vars in 1099 bytes} [Sat Aug 6 21:52:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:52:35.276880 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9f0c45-f595-4d95-bb75-3be0b7e990ed None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:52:35.337028 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:52:35.375382 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9f0c45-f595-4d95-bb75-3be0b7e990ed None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:52:35.408450 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a7a5de-ad01-4db0-900b-8673025a57ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:52:35.409058 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a7a5de-ad01-4db0-900b-8673025a57ab None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:35.409332 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a7a5de-ad01-4db0-900b-8673025a57ab None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:35.409599 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a7a5de-ad01-4db0-900b-8673025a57ab None admin] PATH_INFO: `/v3/users/2f72c036028949e684d050734fcdc812` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:35.410689 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a7a5de-ad01-4db0-900b-8673025a57ab None admin] RBAC: Authorizing `identity:get_user(user_id=2f72c036028949e684d050734fcdc812)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:52:35.412107 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a7a5de-ad01-4db0-900b-8673025a57ab None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:52:35.413671 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 594/1190] 10.222.0.18 () {56 vars in 1177 bytes} [Sat Aug 6 21:52:35 2022] GET /identity/v3/users/2f72c036028949e684d050734fcdc812 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:52:35.420143 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7ee603-219f-4d6c-99c9-6c12e2760af5 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:52:35.517565 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7ee603-219f-4d6c-99c9-6c12e2760af5 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:52:35.549223 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9bc6f7-2215-4200-a21d-9b9c704b38ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:52:35.549828 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9bc6f7-2215-4200-a21d-9b9c704b38ff None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:35.550102 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9bc6f7-2215-4200-a21d-9b9c704b38ff None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:35.550369 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9bc6f7-2215-4200-a21d-9b9c704b38ff None admin] PATH_INFO: `/v3/projects/45f3dac2007b4895a1040d2ddad4be03` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:35.551465 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9bc6f7-2215-4200-a21d-9b9c704b38ff None admin] RBAC: Authorizing `identity:get_project(project_id=45f3dac2007b4895a1040d2ddad4be03)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:52:35.552909 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9bc6f7-2215-4200-a21d-9b9c704b38ff None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:52:35.554483 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 597/1191] 10.222.0.18 () {56 vars in 1186 bytes} [Sat Aug 6 21:52:35 2022] GET /identity/v3/projects/45f3dac2007b4895a1040d2ddad4be03 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:52:35.560543 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a7a5de-ad01-4db0-900b-8673025a57ab None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:52:35.656998 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a7a5de-ad01-4db0-900b-8673025a57ab None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:52:35.685220 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:52:35.689956 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c58219f-5310-446a-813d-4d3ecb0c29d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:52:35.690597 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c58219f-5310-446a-813d-4d3ecb0c29d4 None admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:35.690872 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c58219f-5310-446a-813d-4d3ecb0c29d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:35.691139 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c58219f-5310-446a-813d-4d3ecb0c29d4 None admin] PATH_INFO: `/v3/projects/45f3dac2007b4895a1040d2ddad4be03/users/2f72c036028949e684d050734fcdc812/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:35.693016 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c58219f-5310-446a-813d-4d3ecb0c29d4 None admin] RBAC: Authorizing `identity:create_grant(project_id=45f3dac2007b4895a1040d2ddad4be03, user_id=2f72c036028949e684d050734fcdc812, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:52:35.694431 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c58219f-5310-446a-813d-4d3ecb0c29d4 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:52:35.786091 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 595/1192] 10.222.0.18 () {58 vars in 1439 bytes} [Sat Aug 6 21:52:35 2022] PUT /identity/v3/projects/45f3dac2007b4895a1040d2ddad4be03/users/2f72c036028949e684d050734fcdc812/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 21:52:37.750172 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf43f96d-5f8d-47e1-94e3-86ef78a7b409 None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:37.750720 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf43f96d-5f8d-47e1-94e3-86ef78a7b409 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:37.750720 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf43f96d-5f8d-47e1-94e3-86ef78a7b409 None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:37.752046 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 598/1193] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:52:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:52:37.757144 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1062c614-a743-4d3f-b1c0-17745781629b None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:37.757534 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1062c614-a743-4d3f-b1c0-17745781629b None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:37.757700 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1062c614-a743-4d3f-b1c0-17745781629b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:37.776589 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-1062c614-a743-4d3f-b1c0-17745781629b None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:52:37.777738 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-1062c614-a743-4d3f-b1c0-17745781629b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:52:37.842378 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 596/1194] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:52:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 21:52:37.865204 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf43f96d-5f8d-47e1-94e3-86ef78a7b409 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:52:37.893251 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:52:37.964688 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf43f96d-5f8d-47e1-94e3-86ef78a7b409 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:52:37.997395 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df25e65-f472-44bd-870e-40ec3deb2669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:52:37.998011 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df25e65-f472-44bd-870e-40ec3deb2669 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:37.998287 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df25e65-f472-44bd-870e-40ec3deb2669 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:37.998553 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df25e65-f472-44bd-870e-40ec3deb2669 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:38.008435 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df25e65-f472-44bd-870e-40ec3deb2669 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:52:38.013617 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-5df25e65-f472-44bd-870e-40ec3deb2669 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 21:52:38.013617 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:52:38.013617 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:52:38.013617 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:52:38.013617 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:52:38.013617 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:52:38.013617 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:52:38.013617 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:52:38.013617 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:52:38.013617 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:52:38.013617 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:52:38.013617 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:52:38.013617 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:52:38.013617 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:52:38.013617 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:52:38.013617 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:52:38.014760 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:52:38.014760 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:52:38.014760 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:52:38.014760 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:52:38.014760 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:52:38.014760 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:52:38.014760 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:52:38.014760 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:52:38.014760 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:52:38.014760 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:52:38.014760 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:52:38.014760 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:52:38.014760 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:52:38.014760 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:52:38.014760 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:52:38.014760 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:52:38.015953 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:52:38.015953 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:52:38.015953 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:52:38.015953 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:52:38.015953 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:52:38.015953 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:52:38.015953 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:52:38.015953 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:52:38.015953 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:52:38.015953 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:52:38.015953 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 21:52:38.015953 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:52:38.015953 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 599/1195] 10.222.0.18 () {56 vars in 1096 bytes} [Sat Aug 6 21:52:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:52:38.020656 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1062c614-a743-4d3f-b1c0-17745781629b None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:52:38.116705 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1062c614-a743-4d3f-b1c0-17745781629b None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:52:38.139900 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:52:38.150818 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9081f96a-417c-4a86-bb99-cc2d0185fa20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:52:38.151423 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9081f96a-417c-4a86-bb99-cc2d0185fa20 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:38.151693 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9081f96a-417c-4a86-bb99-cc2d0185fa20 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:38.151961 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9081f96a-417c-4a86-bb99-cc2d0185fa20 None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:38.152637 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9081f96a-417c-4a86-bb99-cc2d0185fa20 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:52:38.152938 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9081f96a-417c-4a86-bb99-cc2d0185fa20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:52:38.154299 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9081f96a-417c-4a86-bb99-cc2d0185fa20 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:52:38.161961 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 597/1196] 10.222.0.18 () {56 vars in 1099 bytes} [Sat Aug 6 21:52:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:52:38.167984 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df25e65-f472-44bd-870e-40ec3deb2669 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:52:38.254820 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:52:38.266832 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df25e65-f472-44bd-870e-40ec3deb2669 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:52:38.299108 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5071fd60-88b3-4d6a-a802-d6822aa689f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:52:38.299723 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5071fd60-88b3-4d6a-a802-d6822aa689f9 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:38.300000 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5071fd60-88b3-4d6a-a802-d6822aa689f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:38.300272 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5071fd60-88b3-4d6a-a802-d6822aa689f9 None admin] PATH_INFO: `/v3/users/2f72c036028949e684d050734fcdc812` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:38.301400 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5071fd60-88b3-4d6a-a802-d6822aa689f9 None admin] RBAC: Authorizing `identity:get_user(user_id=2f72c036028949e684d050734fcdc812)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:52:38.302828 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5071fd60-88b3-4d6a-a802-d6822aa689f9 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:52:38.304379 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 600/1197] 10.222.0.18 () {56 vars in 1177 bytes} [Sat Aug 6 21:52:38 2022] GET /identity/v3/users/2f72c036028949e684d050734fcdc812 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:52:38.310614 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9081f96a-417c-4a86-bb99-cc2d0185fa20 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:52:38.407882 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9081f96a-417c-4a86-bb99-cc2d0185fa20 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:52:38.440031 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a944733-2735-40bc-85cd-bba066bbc973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:52:38.440650 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a944733-2735-40bc-85cd-bba066bbc973 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:38.440930 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a944733-2735-40bc-85cd-bba066bbc973 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:38.441203 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a944733-2735-40bc-85cd-bba066bbc973 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:38.491212 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:52:38.506684 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a944733-2735-40bc-85cd-bba066bbc973 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:52:38.510482 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a944733-2735-40bc-85cd-bba066bbc973 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:52:38.602800 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 598/1198] 10.222.0.18 () {58 vars in 1282 bytes} [Sat Aug 6 21:52:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 21:52:38.609371 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5071fd60-88b3-4d6a-a802-d6822aa689f9 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:52:38.705017 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5071fd60-88b3-4d6a-a802-d6822aa689f9 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:52:38.738160 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7338d27-fc65-451b-b7c5-5d60ae808e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:52:38.738764 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7338d27-fc65-451b-b7c5-5d60ae808e66 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:38.739042 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7338d27-fc65-451b-b7c5-5d60ae808e66 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:38.739316 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7338d27-fc65-451b-b7c5-5d60ae808e66 None admin] PATH_INFO: `/v3/projects/45f3dac2007b4895a1040d2ddad4be03` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:38.740435 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7338d27-fc65-451b-b7c5-5d60ae808e66 None admin] RBAC: Authorizing `identity:get_project(project_id=45f3dac2007b4895a1040d2ddad4be03)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:52:38.741854 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7338d27-fc65-451b-b7c5-5d60ae808e66 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:52:38.743443 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 601/1199] 10.222.0.18 () {56 vars in 1186 bytes} [Sat Aug 6 21:52:38 2022] GET /identity/v3/projects/45f3dac2007b4895a1040d2ddad4be03 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:52:38.749970 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a944733-2735-40bc-85cd-bba066bbc973 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:52:38.829608 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:52:38.844922 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a944733-2735-40bc-85cd-bba066bbc973 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:52:38.877979 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a616f6-44ed-4be8-95d0-3f58e53036aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:52:38.878501 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a616f6-44ed-4be8-95d0-3f58e53036aa None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:52:38.878776 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a616f6-44ed-4be8-95d0-3f58e53036aa None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:52:38.879042 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a616f6-44ed-4be8-95d0-3f58e53036aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:52:38.879753 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a616f6-44ed-4be8-95d0-3f58e53036aa None admin] RBAC: Adding query filter params (role.id=4b8dcf0491fd4072acc69fd894bf77d0, scope.project.id=45f3dac2007b4895a1040d2ddad4be03, user.id=2f72c036028949e684d050734fcdc812) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:52:38.880050 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a616f6-44ed-4be8-95d0-3f58e53036aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:52:38.881482 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a616f6-44ed-4be8-95d0-3f58e53036aa None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:52:38.887035 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 599/1200] 10.222.0.18 () {56 vars in 1374 bytes} [Sat Aug 6 21:52:38 2022] GET /identity/v3/role_assignments?user.id=2f72c036028949e684d050734fcdc812&scope.project.id=45f3dac2007b4895a1040d2ddad4be03&role.id=4b8dcf0491fd4072acc69fd894bf77d0 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:52:48.734068 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=88 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:52:48.884531 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=26 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:53:52.670322 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8c7714-c6a5-4089-9fde-8d0f58f44634 None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:53:52.670876 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8c7714-c6a5-4089-9fde-8d0f58f44634 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:53:52.670876 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8c7714-c6a5-4089-9fde-8d0f58f44634 None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:53:52.672128 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 602/1201] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:53:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:53:52.677352 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a014e9c-18b4-4268-aff6-511759591ece None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:53:52.678420 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a014e9c-18b4-4268-aff6-511759591ece None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:53:52.678420 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a014e9c-18b4-4268-aff6-511759591ece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:53:52.696264 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-3a014e9c-18b4-4268-aff6-511759591ece None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:53:52.697507 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-3a014e9c-18b4-4268-aff6-511759591ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:53:52.765384 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 600/1202] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:53:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 21:53:52.790397 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8c7714-c6a5-4089-9fde-8d0f58f44634 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:53:52.890229 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8c7714-c6a5-4089-9fde-8d0f58f44634 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:53:52.924458 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9612318-a561-4b99-a01c-a54aea3a0347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:53:52.925103 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9612318-a561-4b99-a01c-a54aea3a0347 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:53:52.925384 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9612318-a561-4b99-a01c-a54aea3a0347 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:53:52.925713 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9612318-a561-4b99-a01c-a54aea3a0347 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:53:52.936303 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9612318-a561-4b99-a01c-a54aea3a0347 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:53:52.938085 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9612318-a561-4b99-a01c-a54aea3a0347 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:53:52.952561 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-e9612318-a561-4b99-a01c-a54aea3a0347 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:53:52.952561 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:53:52.952561 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:53:52.952561 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:53:52.952561 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:53:52.952561 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:53:52.952561 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:53:52.952561 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:53:52.952561 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:53:52.952561 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:53:52.952561 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:53:52.952561 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:53:52.952561 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:53:52.952561 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:53:52.952561 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:53:52.952561 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:53:52.953759 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:53:52.953759 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:53:52.953759 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:53:52.953759 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:53:52.953759 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:53:52.953759 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:53:52.953759 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:53:52.953759 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:53:52.953759 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:53:52.953759 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:53:52.953759 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:53:52.953759 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:53:52.953759 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:53:52.953759 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:53:52.953759 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:53:52.953759 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:53:52.953759 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:53:52.954950 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:53:52.954950 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:53:52.954950 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:53:52.954950 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 603/1203] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:53:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:53:52.959959 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a014e9c-18b4-4268-aff6-511759591ece None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:53:53.063427 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a014e9c-18b4-4268-aff6-511759591ece None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:53:53.091123 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:53:53.096684 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de89a363-34cc-4711-a3f3-59c03e4014b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:53:53.097277 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de89a363-34cc-4711-a3f3-59c03e4014b3 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:53:53.097550 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de89a363-34cc-4711-a3f3-59c03e4014b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:53:53.097814 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de89a363-34cc-4711-a3f3-59c03e4014b3 None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:53:53.098533 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de89a363-34cc-4711-a3f3-59c03e4014b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:53:53.098735 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de89a363-34cc-4711-a3f3-59c03e4014b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:53:53.100200 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de89a363-34cc-4711-a3f3-59c03e4014b3 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:53:53.111952 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 601/1204] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:53:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:53:53.117716 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9612318-a561-4b99-a01c-a54aea3a0347 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:53:53.171055 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:53:53.218291 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9612318-a561-4b99-a01c-a54aea3a0347 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:53:53.251424 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b45676-dc28-46ee-bf6a-905df71c811d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:53:53.257944 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b45676-dc28-46ee-bf6a-905df71c811d None admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:53:53.257944 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b45676-dc28-46ee-bf6a-905df71c811d None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:53:53.257944 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b45676-dc28-46ee-bf6a-905df71c811d None admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:53:53.257944 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b45676-dc28-46ee-bf6a-905df71c811d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:53:53.257944 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b45676-dc28-46ee-bf6a-905df71c811d None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:53:53.420037 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 604/1205] 10.222.0.18 () {60 vars in 1132 bytes} [Sat Aug 6 21:53:53 2022] POST /identity/v3/users => generated 251 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 21:53:55.438280 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946df0f0-10e1-4622-9fe1-37265a3cd8f6 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:53:55.438834 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946df0f0-10e1-4622-9fe1-37265a3cd8f6 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:53:55.438834 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946df0f0-10e1-4622-9fe1-37265a3cd8f6 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:53:55.440147 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 602/1206] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:53:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:53:55.446278 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5128f4-08dc-4a04-8025-36c3bde8763d None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:53:55.447643 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5128f4-08dc-4a04-8025-36c3bde8763d None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:53:55.447643 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5128f4-08dc-4a04-8025-36c3bde8763d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:53:55.464601 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-9e5128f4-08dc-4a04-8025-36c3bde8763d None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:53:55.465774 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-9e5128f4-08dc-4a04-8025-36c3bde8763d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:53:55.530732 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 605/1207] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:53:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 21:53:55.554449 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946df0f0-10e1-4622-9fe1-37265a3cd8f6 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:53:55.653982 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946df0f0-10e1-4622-9fe1-37265a3cd8f6 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:53:55.687965 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb6b7b4-a4fc-4b2a-a79c-ae0ab87d9713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:53:55.688611 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb6b7b4-a4fc-4b2a-a79c-ae0ab87d9713 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:53:55.688890 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb6b7b4-a4fc-4b2a-a79c-ae0ab87d9713 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:53:55.689161 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb6b7b4-a4fc-4b2a-a79c-ae0ab87d9713 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:53:55.699924 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb6b7b4-a4fc-4b2a-a79c-ae0ab87d9713 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:53:55.704544 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-5bb6b7b4-a4fc-4b2a-a79c-ae0ab87d9713 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 21:53:55.704544 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:53:55.704544 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:53:55.704544 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:53:55.704544 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:53:55.704544 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:53:55.704544 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:53:55.704544 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:53:55.704544 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:53:55.704544 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:53:55.704544 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:53:55.704544 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:53:55.704544 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:53:55.704544 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:53:55.704544 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:53:55.704544 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:53:55.705744 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:53:55.705744 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:53:55.705744 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:53:55.705744 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:53:55.705744 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:53:55.705744 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:53:55.705744 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:53:55.705744 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:53:55.705744 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:53:55.705744 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:53:55.705744 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:53:55.705744 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:53:55.705744 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:53:55.705744 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:53:55.705744 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:53:55.705744 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:53:55.706784 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:53:55.706784 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:53:55.706784 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:53:55.706784 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:53:55.706784 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:53:55.706784 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:53:55.706784 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:53:55.706784 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:53:55.706784 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:53:55.706784 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:53:55.706784 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 21:53:55.706784 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:53:55.706784 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 603/1208] 10.222.0.18 () {56 vars in 1102 bytes} [Sat Aug 6 21:53:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:53:55.711760 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5128f4-08dc-4a04-8025-36c3bde8763d None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:53:55.750448 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:53:55.806862 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5128f4-08dc-4a04-8025-36c3bde8763d None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:53:55.839832 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04927d9-a215-4f16-9d10-8f5462c5e39f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:53:55.840439 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04927d9-a215-4f16-9d10-8f5462c5e39f None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:53:55.840718 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04927d9-a215-4f16-9d10-8f5462c5e39f None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:53:55.840986 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04927d9-a215-4f16-9d10-8f5462c5e39f None admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:53:55.841642 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04927d9-a215-4f16-9d10-8f5462c5e39f None admin] RBAC: Adding query filter params (name=service) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:53:55.841940 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04927d9-a215-4f16-9d10-8f5462c5e39f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:53:55.843286 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04927d9-a215-4f16-9d10-8f5462c5e39f None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:53:55.851069 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 606/1209] 10.222.0.18 () {56 vars in 1103 bytes} [Sat Aug 6 21:53:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:53:55.857192 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb6b7b4-a4fc-4b2a-a79c-ae0ab87d9713 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:53:55.906422 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:53:55.955007 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb6b7b4-a4fc-4b2a-a79c-ae0ab87d9713 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:53:55.988153 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4ae2b7-d66d-45e8-84b4-01a92f804fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:53:55.988797 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4ae2b7-d66d-45e8-84b4-01a92f804fbd None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:53:55.989078 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4ae2b7-d66d-45e8-84b4-01a92f804fbd None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:53:55.989349 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4ae2b7-d66d-45e8-84b4-01a92f804fbd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:53:55.999833 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4ae2b7-d66d-45e8-84b4-01a92f804fbd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:53:56.001395 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4ae2b7-d66d-45e8-84b4-01a92f804fbd None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:53:56.015816 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-8d4ae2b7-d66d-45e8-84b4-01a92f804fbd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:53:56.015816 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:53:56.015816 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:53:56.015816 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:53:56.015816 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:53:56.015816 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:53:56.015816 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:53:56.015816 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:53:56.015816 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:53:56.015816 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:53:56.015816 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:53:56.015816 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:53:56.015816 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:53:56.015816 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:53:56.015816 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:53:56.015816 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:53:56.016984 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:53:56.016984 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:53:56.016984 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:53:56.016984 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:53:56.016984 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:53:56.016984 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:53:56.016984 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:53:56.016984 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:53:56.016984 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:53:56.016984 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:53:56.016984 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:53:56.016984 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:53:56.016984 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:53:56.016984 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:53:56.016984 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:53:56.016984 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:53:56.016984 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:53:56.018349 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:53:56.018349 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:53:56.018349 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:53:56.018349 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 604/1210] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:53:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:53:56.023360 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04927d9-a215-4f16-9d10-8f5462c5e39f None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:53:56.118653 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:53:56.121579 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04927d9-a215-4f16-9d10-8f5462c5e39f None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:53:56.154283 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1500b0d-dbd2-44d2-a58a-0c8551957d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:53:56.154875 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1500b0d-dbd2-44d2-a58a-0c8551957d52 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:53:56.155193 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1500b0d-dbd2-44d2-a58a-0c8551957d52 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:53:56.155460 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1500b0d-dbd2-44d2-a58a-0c8551957d52 None admin] PATH_INFO: `/v3/domains` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:53:56.156093 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1500b0d-dbd2-44d2-a58a-0c8551957d52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:53:56.156480 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1500b0d-dbd2-44d2-a58a-0c8551957d52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:53:56.157777 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1500b0d-dbd2-44d2-a58a-0c8551957d52 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:53:56.168698 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 607/1211] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:53:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 21:53:56.174425 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4ae2b7-d66d-45e8-84b4-01a92f804fbd None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:53:56.273801 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4ae2b7-d66d-45e8-84b4-01a92f804fbd None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:53:56.275314 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:53:56.307168 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acb7935-d667-47e7-9446-3b6d6149a4d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:53:56.307786 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acb7935-d667-47e7-9446-3b6d6149a4d8 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:53:56.308064 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acb7935-d667-47e7-9446-3b6d6149a4d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:53:56.308335 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acb7935-d667-47e7-9446-3b6d6149a4d8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:53:56.316061 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acb7935-d667-47e7-9446-3b6d6149a4d8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:53:56.319262 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acb7935-d667-47e7-9446-3b6d6149a4d8 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:53:56.330032 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-1acb7935-d667-47e7-9446-3b6d6149a4d8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 06 21:53:56.330032 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:53:56.330032 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:53:56.330032 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:53:56.330032 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:53:56.330032 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:53:56.330032 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:53:56.330032 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:53:56.330032 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:53:56.330032 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:53:56.330032 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:53:56.330032 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:53:56.330032 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 21:53:56.330032 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 21:53:56.330032 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 21:53:56.330032 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 21:53:56.331191 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:53:56.331191 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:53:56.331191 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 21:53:56.331191 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 21:53:56.331191 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 21:53:56.331191 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 21:53:56.331191 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:53:56.331191 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:53:56.331191 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:53:56.331191 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:53:56.331191 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:53:56.331191 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:53:56.331191 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:53:56.331191 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:53:56.331191 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:53:56.331191 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:53:56.331191 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:53:56.331191 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:53:56.332469 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:53:56.332469 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:53:56.332469 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 21:53:56.332469 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 21:53:56.332469 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 21:53:56.332469 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 21:53:56.332469 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 21:53:56.332469 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 21:53:56.332469 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 06 21:53:56.332469 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:53:56.332469 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 605/1212] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:53:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:53:56.337491 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1500b0d-dbd2-44d2-a58a-0c8551957d52 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:53:56.437522 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1500b0d-dbd2-44d2-a58a-0c8551957d52 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:53:56.471155 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325bc6b2-7514-4e6a-ad83-1f8075e8b9a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:53:56.471757 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325bc6b2-7514-4e6a-ad83-1f8075e8b9a6 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:53:56.472037 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325bc6b2-7514-4e6a-ad83-1f8075e8b9a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:53:56.472311 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325bc6b2-7514-4e6a-ad83-1f8075e8b9a6 None admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:53:56.473187 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325bc6b2-7514-4e6a-ad83-1f8075e8b9a6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:53:56.473497 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325bc6b2-7514-4e6a-ad83-1f8075e8b9a6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:53:56.475530 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325bc6b2-7514-4e6a-ad83-1f8075e8b9a6 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:53:56.493326 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 608/1213] 10.222.0.18 () {56 vars in 1143 bytes} [Sat Aug 6 21:53:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:53:56.499157 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acb7935-d667-47e7-9446-3b6d6149a4d8 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:53:56.598739 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acb7935-d667-47e7-9446-3b6d6149a4d8 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:53:56.632820 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00ad8b0-cfa0-4f5e-8e86-ff2943f3434a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:53:56.633410 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00ad8b0-cfa0-4f5e-8e86-ff2943f3434a None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:53:56.633686 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00ad8b0-cfa0-4f5e-8e86-ff2943f3434a None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:53:56.633953 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00ad8b0-cfa0-4f5e-8e86-ff2943f3434a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:53:56.662103 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:53:56.699810 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00ad8b0-cfa0-4f5e-8e86-ff2943f3434a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:53:56.701235 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00ad8b0-cfa0-4f5e-8e86-ff2943f3434a None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:53:56.798139 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 606/1214] 10.222.0.18 () {58 vars in 1282 bytes} [Sat Aug 6 21:53:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 21:53:56.804683 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325bc6b2-7514-4e6a-ad83-1f8075e8b9a6 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:53:56.822940 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:53:56.903195 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325bc6b2-7514-4e6a-ad83-1f8075e8b9a6 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:53:56.936802 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2ee075-3d90-48cf-8e8d-2e922f40706e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:53:56.937418 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2ee075-3d90-48cf-8e8d-2e922f40706e None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:53:56.937696 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2ee075-3d90-48cf-8e8d-2e922f40706e None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:53:56.937963 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2ee075-3d90-48cf-8e8d-2e922f40706e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:53:56.948385 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2ee075-3d90-48cf-8e8d-2e922f40706e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:53:56.954094 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2ee075-3d90-48cf-8e8d-2e922f40706e None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:53:56.964185 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-ee2ee075-3d90-48cf-8e8d-2e922f40706e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:53:56.964185 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:53:56.964185 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:53:56.964185 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:53:56.964185 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:53:56.964185 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:53:56.964185 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:53:56.964185 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:53:56.964185 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:53:56.964185 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:53:56.964185 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:53:56.964185 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:53:56.964185 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:53:56.964185 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:53:56.964185 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:53:56.964185 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:53:56.965317 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:53:56.965317 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:53:56.965317 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:53:56.965317 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:53:56.965317 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:53:56.965317 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:53:56.965317 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:53:56.965317 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:53:56.965317 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:53:56.965317 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:53:56.965317 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:53:56.965317 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:53:56.965317 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:53:56.965317 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:53:56.965317 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:53:56.965317 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:53:56.965317 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:53:56.966451 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:53:56.966451 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:53:56.966451 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:53:56.966451 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 609/1215] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:53:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:53:56.971773 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00ad8b0-cfa0-4f5e-8e86-ff2943f3434a None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:53:57.031866 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:53:57.070660 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00ad8b0-cfa0-4f5e-8e86-ff2943f3434a None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:53:57.103957 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca24884-4f9b-4e5d-8203-237e3842e046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:53:57.104685 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca24884-4f9b-4e5d-8203-237e3842e046 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:53:57.104959 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca24884-4f9b-4e5d-8203-237e3842e046 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:53:57.105228 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca24884-4f9b-4e5d-8203-237e3842e046 None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:53:57.105863 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca24884-4f9b-4e5d-8203-237e3842e046 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:53:57.106191 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca24884-4f9b-4e5d-8203-237e3842e046 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:53:57.107820 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca24884-4f9b-4e5d-8203-237e3842e046 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:53:57.119355 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 607/1216] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:53:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:53:57.124954 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2ee075-3d90-48cf-8e8d-2e922f40706e None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:53:57.192666 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:53:57.224110 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2ee075-3d90-48cf-8e8d-2e922f40706e None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:53:57.258155 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f2298e-e298-4034-9b04-e59c10a59971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:53:57.258773 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f2298e-e298-4034-9b04-e59c10a59971 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:53:57.259476 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f2298e-e298-4034-9b04-e59c10a59971 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:53:57.259476 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f2298e-e298-4034-9b04-e59c10a59971 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:53:57.265738 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f2298e-e298-4034-9b04-e59c10a59971 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:53:57.267173 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f2298e-e298-4034-9b04-e59c10a59971 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:53:57.278243 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-95f2298e-e298-4034-9b04-e59c10a59971 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 21:53:57.278243 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:53:57.278243 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:53:57.278243 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:53:57.278243 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:53:57.278243 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:53:57.278243 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:53:57.278243 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:53:57.278243 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:53:57.278243 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:53:57.278243 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:53:57.278243 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:53:57.278243 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 21:53:57.278243 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 21:53:57.278243 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 21:53:57.279298 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 21:53:57.279298 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:53:57.279298 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:53:57.279298 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:53:57.279298 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:53:57.279298 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:53:57.279298 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:53:57.279298 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:53:57.279298 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:53:57.279298 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:53:57.279298 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:53:57.279298 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:53:57.279298 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:53:57.279298 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:53:57.279298 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:53:57.279298 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:53:57.279298 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:53:57.280705 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 21:53:57.280705 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 21:53:57.280705 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 21:53:57.280705 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 21:53:57.280705 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 21:53:57.280705 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 21:53:57.280705 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 21:53:57.280705 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:53:57.280705 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 610/1217] 10.222.0.18 () {56 vars in 1111 bytes} [Sat Aug 6 21:53:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:53:57.285247 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca24884-4f9b-4e5d-8203-237e3842e046 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:53:57.386146 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca24884-4f9b-4e5d-8203-237e3842e046 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:53:57.407767 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:53:57.420578 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52412f85-8460-4dba-bcf6-76b646ad59e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:53:57.421182 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52412f85-8460-4dba-bcf6-76b646ad59e4 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:53:57.421463 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52412f85-8460-4dba-bcf6-76b646ad59e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:53:57.421735 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52412f85-8460-4dba-bcf6-76b646ad59e4 None admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:53:57.422416 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52412f85-8460-4dba-bcf6-76b646ad59e4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:53:57.422760 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52412f85-8460-4dba-bcf6-76b646ad59e4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:53:57.424194 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52412f85-8460-4dba-bcf6-76b646ad59e4 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:53:57.437173 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 608/1218] 10.222.0.18 () {56 vars in 1148 bytes} [Sat Aug 6 21:53:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:53:57.443331 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f2298e-e298-4034-9b04-e59c10a59971 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:53:57.542741 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f2298e-e298-4034-9b04-e59c10a59971 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:53:57.570452 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:53:57.576081 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fde54c-266a-4c4e-850b-2085a035f5b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:53:57.576716 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fde54c-266a-4c4e-850b-2085a035f5b5 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:53:57.576990 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fde54c-266a-4c4e-850b-2085a035f5b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:53:57.577260 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fde54c-266a-4c4e-850b-2085a035f5b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:53:57.577973 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fde54c-266a-4c4e-850b-2085a035f5b5 None admin] RBAC: Adding query filter params (role.id=d90649d4ef604eeb8edcf0aa22932ca1, scope.project.id=4bb6307eb0644d85a1d1daaa61352697, user.id=680d62a67ae2456f8c129e26728c3297) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:53:57.578273 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fde54c-266a-4c4e-850b-2085a035f5b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:53:57.579681 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fde54c-266a-4c4e-850b-2085a035f5b5 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:53:57.585473 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 611/1219] 10.222.0.18 () {56 vars in 1374 bytes} [Sat Aug 6 21:53:57 2022] GET /identity/v3/role_assignments?user.id=680d62a67ae2456f8c129e26728c3297&scope.project.id=4bb6307eb0644d85a1d1daaa61352697&role.id=d90649d4ef604eeb8edcf0aa22932ca1 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:53:59.570439 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0249c853-9413-4246-8933-b0c1cf8e6f90 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:53:59.571077 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0249c853-9413-4246-8933-b0c1cf8e6f90 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:53:59.571077 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0249c853-9413-4246-8933-b0c1cf8e6f90 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:53:59.572297 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 609/1220] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:53:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 21:53:59.578122 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a68681-8849-43bc-99f7-6700888ffba8 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:53:59.581029 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a68681-8849-43bc-99f7-6700888ffba8 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:53:59.581029 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a68681-8849-43bc-99f7-6700888ffba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:53:59.596807 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-06a68681-8849-43bc-99f7-6700888ffba8 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:53:59.597966 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-06a68681-8849-43bc-99f7-6700888ffba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:53:59.662417 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 612/1221] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:53:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 21:53:59.684472 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0249c853-9413-4246-8933-b0c1cf8e6f90 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:53:59.786790 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0249c853-9413-4246-8933-b0c1cf8e6f90 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:53:59.821024 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b38862-0b28-452a-844d-2f6ba3175d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:53:59.821642 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b38862-0b28-452a-844d-2f6ba3175d64 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:53:59.821920 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b38862-0b28-452a-844d-2f6ba3175d64 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:53:59.822193 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b38862-0b28-452a-844d-2f6ba3175d64 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:53:59.832045 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b38862-0b28-452a-844d-2f6ba3175d64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:53:59.836737 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-67b38862-0b28-452a-844d-2f6ba3175d64 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 21:53:59.836737 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:53:59.836737 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:53:59.836737 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:53:59.836737 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:53:59.836737 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:53:59.836737 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:53:59.836737 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:53:59.836737 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:53:59.836737 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:53:59.836737 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:53:59.836737 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:53:59.836737 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:53:59.836737 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:53:59.836737 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:53:59.836737 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:53:59.843369 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:53:59.843369 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:53:59.843369 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:53:59.843369 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:53:59.843369 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:53:59.843369 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:53:59.843369 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:53:59.843369 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:53:59.843369 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:53:59.843369 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:53:59.843369 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:53:59.843369 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:53:59.843369 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:53:59.843369 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:53:59.843369 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:53:59.843369 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:53:59.844621 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:53:59.844621 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:53:59.844621 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:53:59.844621 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:53:59.844621 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:53:59.844621 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:53:59.844621 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:53:59.844621 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:53:59.844621 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:53:59.844621 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:53:59.844621 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 21:53:59.844621 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:53:59.844621 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 610/1222] 10.222.0.18 () {56 vars in 1102 bytes} [Sat Aug 6 21:53:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:53:59.845601 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a68681-8849-43bc-99f7-6700888ffba8 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:53:59.942031 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a68681-8849-43bc-99f7-6700888ffba8 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:53:59.965190 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:53:59.975552 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a846a4-4f45-47c0-915f-682346770ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:53:59.976143 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a846a4-4f45-47c0-915f-682346770ead None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:53:59.976443 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a846a4-4f45-47c0-915f-682346770ead None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:53:59.976716 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a846a4-4f45-47c0-915f-682346770ead None admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:53:59.977374 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a846a4-4f45-47c0-915f-682346770ead None admin] RBAC: Adding query filter params (name=service) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:53:59.977691 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a846a4-4f45-47c0-915f-682346770ead None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:53:59.979044 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a846a4-4f45-47c0-915f-682346770ead None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:53:59.987183 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 613/1223] 10.222.0.18 () {56 vars in 1103 bytes} [Sat Aug 6 21:53:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:53:59.993904 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b38862-0b28-452a-844d-2f6ba3175d64 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:54:00.038168 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:54:00.092030 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b38862-0b28-452a-844d-2f6ba3175d64 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:54:00.130183 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcca5472-f62a-4d4b-a3d5-f1b7cd4605c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:54:00.131525 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcca5472-f62a-4d4b-a3d5-f1b7cd4605c4 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:00.131808 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcca5472-f62a-4d4b-a3d5-f1b7cd4605c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:00.132080 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcca5472-f62a-4d4b-a3d5-f1b7cd4605c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:00.142593 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcca5472-f62a-4d4b-a3d5-f1b7cd4605c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:54:00.144036 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcca5472-f62a-4d4b-a3d5-f1b7cd4605c4 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:54:00.158514 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-bcca5472-f62a-4d4b-a3d5-f1b7cd4605c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:54:00.158514 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:54:00.158514 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:54:00.158514 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:54:00.158514 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:54:00.158514 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:54:00.158514 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:54:00.158514 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:54:00.158514 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:54:00.158514 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:54:00.158514 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:54:00.158514 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:54:00.158514 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:54:00.158514 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:54:00.158514 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:54:00.158514 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:54:00.161410 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:54:00.161410 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:54:00.161410 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:54:00.161410 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:54:00.161410 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:54:00.161410 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:54:00.161410 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:54:00.161410 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:54:00.161410 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:54:00.161410 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:54:00.161410 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:54:00.161410 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:54:00.161410 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:54:00.161410 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:54:00.161410 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:54:00.161410 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:54:00.161410 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:54:00.162601 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:54:00.162601 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:54:00.162601 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:54:00.162601 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 611/1224] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:53:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:54:00.165879 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a846a4-4f45-47c0-915f-682346770ead None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:54:00.263154 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a846a4-4f45-47c0-915f-682346770ead None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:54:00.295483 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63104065-3ebd-4d80-9c4f-1a76b88aba6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:54:00.296072 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63104065-3ebd-4d80-9c4f-1a76b88aba6b None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:00.296346 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63104065-3ebd-4d80-9c4f-1a76b88aba6b None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:00.296644 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63104065-3ebd-4d80-9c4f-1a76b88aba6b None admin] PATH_INFO: `/v3/domains` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:00.297283 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63104065-3ebd-4d80-9c4f-1a76b88aba6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:54:00.297578 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63104065-3ebd-4d80-9c4f-1a76b88aba6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:54:00.298932 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63104065-3ebd-4d80-9c4f-1a76b88aba6b None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:54:00.311429 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 614/1225] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:54:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:54:00.317209 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcca5472-f62a-4d4b-a3d5-f1b7cd4605c4 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:54:00.413043 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:54:00.419864 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcca5472-f62a-4d4b-a3d5-f1b7cd4605c4 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:54:00.456376 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02b665f-85d7-4439-bc36-2c8d7f6e3b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:54:00.457022 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02b665f-85d7-4439-bc36-2c8d7f6e3b97 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:00.457302 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02b665f-85d7-4439-bc36-2c8d7f6e3b97 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:00.457573 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02b665f-85d7-4439-bc36-2c8d7f6e3b97 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:00.465096 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02b665f-85d7-4439-bc36-2c8d7f6e3b97 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:54:00.466537 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02b665f-85d7-4439-bc36-2c8d7f6e3b97 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:54:00.478201 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-e02b665f-85d7-4439-bc36-2c8d7f6e3b97 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 06 21:54:00.478201 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:54:00.478201 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:54:00.478201 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:54:00.478201 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:54:00.478201 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:54:00.478201 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:54:00.478201 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:54:00.478201 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:54:00.478201 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:54:00.478201 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:54:00.478201 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:54:00.478201 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 21:54:00.478201 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 21:54:00.478201 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 21:54:00.478201 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 21:54:00.479367 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:54:00.479367 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:54:00.479367 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 21:54:00.479367 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 21:54:00.479367 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 21:54:00.479367 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 21:54:00.479367 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:54:00.479367 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:54:00.479367 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:54:00.479367 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:54:00.479367 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:54:00.479367 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:54:00.479367 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:54:00.479367 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:54:00.479367 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:54:00.479367 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:54:00.479367 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:54:00.479367 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:54:00.480626 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:54:00.480626 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:54:00.480626 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 21:54:00.480626 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 21:54:00.480626 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 21:54:00.480626 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 21:54:00.480626 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 21:54:00.480626 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 21:54:00.480626 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 06 21:54:00.480626 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:54:00.480626 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 612/1226] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:54:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:54:00.485555 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63104065-3ebd-4d80-9c4f-1a76b88aba6b None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:54:00.531914 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:54:00.583042 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63104065-3ebd-4d80-9c4f-1a76b88aba6b None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:54:00.616705 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372e8a39-d8e6-4305-8324-c989239e10a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:54:00.617294 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372e8a39-d8e6-4305-8324-c989239e10a3 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:00.617569 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372e8a39-d8e6-4305-8324-c989239e10a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:00.617836 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372e8a39-d8e6-4305-8324-c989239e10a3 None admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:00.618545 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372e8a39-d8e6-4305-8324-c989239e10a3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:54:00.618845 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372e8a39-d8e6-4305-8324-c989239e10a3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:54:00.620245 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372e8a39-d8e6-4305-8324-c989239e10a3 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:54:00.639328 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 615/1227] 10.222.0.18 () {56 vars in 1143 bytes} [Sat Aug 6 21:54:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:54:00.644637 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02b665f-85d7-4439-bc36-2c8d7f6e3b97 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:54:00.743300 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02b665f-85d7-4439-bc36-2c8d7f6e3b97 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:54:00.776644 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d3da39-ea9c-411f-87ff-f545e414de59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:54:00.777263 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d3da39-ea9c-411f-87ff-f545e414de59 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:00.777538 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d3da39-ea9c-411f-87ff-f545e414de59 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:00.777807 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d3da39-ea9c-411f-87ff-f545e414de59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:00.788656 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d3da39-ea9c-411f-87ff-f545e414de59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:54:00.790045 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d3da39-ea9c-411f-87ff-f545e414de59 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:54:00.803427 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-27d3da39-ea9c-411f-87ff-f545e414de59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:54:00.803427 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:54:00.803427 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:54:00.803427 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:54:00.803427 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:54:00.803427 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:54:00.803427 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:54:00.803427 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:54:00.803427 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:54:00.803427 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:54:00.803427 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:54:00.803427 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:54:00.803427 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:54:00.803427 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:54:00.803427 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:54:00.803427 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:54:00.804582 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:54:00.804582 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:54:00.804582 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:54:00.804582 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:54:00.804582 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:54:00.804582 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:54:00.804582 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:54:00.804582 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:54:00.804582 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:54:00.804582 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:54:00.804582 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:54:00.804582 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:54:00.804582 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:54:00.804582 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:54:00.804582 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:54:00.804582 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:54:00.804582 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:54:00.805661 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:54:00.805661 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:54:00.805661 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:54:00.805661 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 613/1228] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:54:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:54:00.811670 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372e8a39-d8e6-4305-8324-c989239e10a3 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:54:00.909760 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372e8a39-d8e6-4305-8324-c989239e10a3 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:54:00.912659 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:54:00.943017 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0607482-50cd-4b1e-bcc8-cce47c99e6c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:54:00.944206 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0607482-50cd-4b1e-bcc8-cce47c99e6c6 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:00.944511 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0607482-50cd-4b1e-bcc8-cce47c99e6c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:00.944782 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0607482-50cd-4b1e-bcc8-cce47c99e6c6 None admin] PATH_INFO: `/v3/domains` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:00.945447 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0607482-50cd-4b1e-bcc8-cce47c99e6c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:54:00.945747 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0607482-50cd-4b1e-bcc8-cce47c99e6c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:54:00.947104 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0607482-50cd-4b1e-bcc8-cce47c99e6c6 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:54:00.958027 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 616/1229] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:54:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:54:00.964493 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d3da39-ea9c-411f-87ff-f545e414de59 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:54:00.973723 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:54:01.060461 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d3da39-ea9c-411f-87ff-f545e414de59 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:54:01.094344 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b24caa-c183-4d66-bc61-ffd4d71ba60f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:54:01.094966 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b24caa-c183-4d66-bc61-ffd4d71ba60f None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:01.095245 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b24caa-c183-4d66-bc61-ffd4d71ba60f None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:01.095516 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b24caa-c183-4d66-bc61-ffd4d71ba60f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:01.102223 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b24caa-c183-4d66-bc61-ffd4d71ba60f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:54:01.103661 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b24caa-c183-4d66-bc61-ffd4d71ba60f None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:54:01.114705 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-e0b24caa-c183-4d66-bc61-ffd4d71ba60f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 21:54:01.114705 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:54:01.114705 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:54:01.114705 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:54:01.114705 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:54:01.114705 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:54:01.114705 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:54:01.114705 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:54:01.114705 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:54:01.114705 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:54:01.114705 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:54:01.114705 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:54:01.114705 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 21:54:01.114705 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 21:54:01.114705 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 21:54:01.115797 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 21:54:01.115797 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:54:01.115797 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:54:01.115797 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:54:01.115797 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:54:01.115797 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:54:01.115797 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:54:01.115797 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:54:01.115797 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:54:01.115797 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:54:01.115797 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:54:01.115797 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:54:01.115797 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:54:01.115797 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:54:01.115797 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:54:01.115797 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:54:01.115797 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:54:01.116941 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 21:54:01.116941 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 21:54:01.116941 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 21:54:01.116941 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 21:54:01.116941 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 21:54:01.116941 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 21:54:01.116941 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 21:54:01.116941 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:54:01.116941 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 614/1230] 10.222.0.18 () {56 vars in 1111 bytes} [Sat Aug 6 21:54:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:54:01.121846 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0607482-50cd-4b1e-bcc8-cce47c99e6c6 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:54:01.220252 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0607482-50cd-4b1e-bcc8-cce47c99e6c6 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:54:01.253768 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684ae558-1b06-4095-a1f4-c66436ffcfd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:54:01.254359 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684ae558-1b06-4095-a1f4-c66436ffcfd6 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:01.254636 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684ae558-1b06-4095-a1f4-c66436ffcfd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:01.254909 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684ae558-1b06-4095-a1f4-c66436ffcfd6 None admin] PATH_INFO: `/v3/projects` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:01.255595 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684ae558-1b06-4095-a1f4-c66436ffcfd6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:54:01.255900 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684ae558-1b06-4095-a1f4-c66436ffcfd6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:54:01.257515 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684ae558-1b06-4095-a1f4-c66436ffcfd6 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:54:01.269194 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 617/1231] 10.222.0.18 () {56 vars in 1148 bytes} [Sat Aug 6 21:54:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:54:01.275303 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b24caa-c183-4d66-bc61-ffd4d71ba60f None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:54:01.341938 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:54:01.377676 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b24caa-c183-4d66-bc61-ffd4d71ba60f None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:54:01.410345 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a64b56-074c-4f87-95e6-0be621457e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:54:01.410990 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a64b56-074c-4f87-95e6-0be621457e7a None admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:01.411267 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a64b56-074c-4f87-95e6-0be621457e7a None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:01.411543 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a64b56-074c-4f87-95e6-0be621457e7a None admin] PATH_INFO: `/v3/projects/4bb6307eb0644d85a1d1daaa61352697/users/680d62a67ae2456f8c129e26728c3297/roles/d90649d4ef604eeb8edcf0aa22932ca1` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:01.426146 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a64b56-074c-4f87-95e6-0be621457e7a None admin] RBAC: Authorizing `identity:create_grant(project_id=4bb6307eb0644d85a1d1daaa61352697, user_id=680d62a67ae2456f8c129e26728c3297, role_id=d90649d4ef604eeb8edcf0aa22932ca1)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:54:01.428198 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a64b56-074c-4f87-95e6-0be621457e7a None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:54:01.575647 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 615/1232] 10.222.0.18 () {58 vars in 1439 bytes} [Sat Aug 6 21:54:01 2022] PUT /identity/v3/projects/4bb6307eb0644d85a1d1daaa61352697/users/680d62a67ae2456f8c129e26728c3297/roles/d90649d4ef604eeb8edcf0aa22932ca1 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 21:54:03.563639 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ef833-6b58-4e94-8683-9f711b5c2dfa None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:03.564191 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ef833-6b58-4e94-8683-9f711b5c2dfa None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:03.564191 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ef833-6b58-4e94-8683-9f711b5c2dfa None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:03.565672 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 618/1233] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:54:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:54:03.570776 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cae791-46ca-44ad-9121-7efad09b735e None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:03.571196 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cae791-46ca-44ad-9121-7efad09b735e None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:03.571325 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cae791-46ca-44ad-9121-7efad09b735e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:03.589771 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-23cae791-46ca-44ad-9121-7efad09b735e None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:54:03.590960 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-23cae791-46ca-44ad-9121-7efad09b735e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:54:03.656156 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 616/1234] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:54:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 21:54:03.679916 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119ef833-6b58-4e94-8683-9f711b5c2dfa None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:54:03.704617 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:54:03.781087 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119ef833-6b58-4e94-8683-9f711b5c2dfa None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:54:03.815743 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabe2ec4-a2e5-4d4c-953d-594751e16959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:54:03.816389 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabe2ec4-a2e5-4d4c-953d-594751e16959 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:03.816734 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabe2ec4-a2e5-4d4c-953d-594751e16959 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:03.828064 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabe2ec4-a2e5-4d4c-953d-594751e16959 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:03.828064 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabe2ec4-a2e5-4d4c-953d-594751e16959 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:54:03.831982 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-dabe2ec4-a2e5-4d4c-953d-594751e16959 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 21:54:03.831982 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:54:03.831982 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:54:03.831982 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:54:03.831982 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:54:03.831982 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:54:03.831982 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:54:03.831982 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:54:03.831982 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:54:03.831982 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:54:03.831982 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:54:03.831982 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:54:03.831982 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:54:03.831982 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:54:03.831982 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:54:03.831982 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:54:03.833167 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:54:03.833167 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:54:03.833167 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:54:03.833167 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:54:03.833167 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:54:03.833167 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:54:03.833167 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:54:03.833167 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:54:03.833167 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:54:03.833167 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:54:03.833167 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:54:03.833167 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:54:03.833167 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:54:03.833167 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:54:03.833167 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:54:03.833167 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:54:03.834900 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:54:03.834900 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:54:03.834900 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:54:03.834900 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:54:03.834900 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:54:03.834900 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:54:03.834900 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:54:03.834900 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:54:03.834900 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:54:03.834900 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:54:03.834900 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 21:54:03.834900 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:54:03.834900 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 619/1235] 10.222.0.18 () {56 vars in 1102 bytes} [Sat Aug 6 21:54:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:54:03.839641 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cae791-46ca-44ad-9121-7efad09b735e None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:54:03.885657 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:54:03.937730 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cae791-46ca-44ad-9121-7efad09b735e None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:54:03.971683 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5439040-4df0-4cb6-9a1b-9b9d80c237ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:54:03.972275 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5439040-4df0-4cb6-9a1b-9b9d80c237ab None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:03.972584 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5439040-4df0-4cb6-9a1b-9b9d80c237ab None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:03.972861 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5439040-4df0-4cb6-9a1b-9b9d80c237ab None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:03.973542 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5439040-4df0-4cb6-9a1b-9b9d80c237ab None admin] RBAC: Adding query filter params (name=service) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:54:03.973838 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5439040-4df0-4cb6-9a1b-9b9d80c237ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:54:03.975232 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5439040-4df0-4cb6-9a1b-9b9d80c237ab None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:54:03.983405 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 617/1236] 10.222.0.18 () {56 vars in 1103 bytes} [Sat Aug 6 21:54:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:54:03.989302 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabe2ec4-a2e5-4d4c-953d-594751e16959 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:54:04.072047 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:54:04.087999 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabe2ec4-a2e5-4d4c-953d-594751e16959 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:54:04.121574 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744509f3-43ea-4977-a556-b5f3bf028823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:54:04.122191 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744509f3-43ea-4977-a556-b5f3bf028823 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:04.122470 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744509f3-43ea-4977-a556-b5f3bf028823 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:04.122741 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744509f3-43ea-4977-a556-b5f3bf028823 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:04.133207 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744509f3-43ea-4977-a556-b5f3bf028823 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:54:04.134631 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744509f3-43ea-4977-a556-b5f3bf028823 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:54:04.149324 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-744509f3-43ea-4977-a556-b5f3bf028823 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:54:04.149324 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:54:04.149324 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:54:04.149324 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:54:04.149324 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:54:04.149324 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:54:04.149324 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:54:04.149324 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:54:04.149324 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:54:04.149324 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:54:04.149324 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:54:04.149324 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:54:04.149324 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:54:04.149324 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:54:04.149324 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:54:04.149324 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:54:04.150418 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:54:04.150418 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:54:04.150418 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:54:04.150418 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:54:04.150418 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:54:04.150418 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:54:04.150418 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:54:04.150418 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:54:04.150418 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:54:04.150418 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:54:04.150418 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:54:04.150418 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:54:04.150418 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:54:04.150418 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:54:04.150418 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:54:04.150418 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:54:04.150418 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:54:04.151324 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:54:04.151324 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:54:04.151324 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:54:04.151324 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 620/1237] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:54:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:54:04.156174 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5439040-4df0-4cb6-9a1b-9b9d80c237ab None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:54:04.255757 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5439040-4df0-4cb6-9a1b-9b9d80c237ab None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:54:04.264620 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:54:04.289015 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d996494-9f51-4327-8d23-b8c53e6d18b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:54:04.289613 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d996494-9f51-4327-8d23-b8c53e6d18b4 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:04.289890 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d996494-9f51-4327-8d23-b8c53e6d18b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:04.290158 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d996494-9f51-4327-8d23-b8c53e6d18b4 None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:04.290788 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d996494-9f51-4327-8d23-b8c53e6d18b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:54:04.291087 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d996494-9f51-4327-8d23-b8c53e6d18b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:54:04.292476 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d996494-9f51-4327-8d23-b8c53e6d18b4 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:54:04.304445 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 618/1238] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:54:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:54:04.310068 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744509f3-43ea-4977-a556-b5f3bf028823 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:54:04.411661 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744509f3-43ea-4977-a556-b5f3bf028823 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:54:04.444861 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f45fb4e-563c-4424-9097-3ca0be993882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:54:04.445512 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f45fb4e-563c-4424-9097-3ca0be993882 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:04.445788 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f45fb4e-563c-4424-9097-3ca0be993882 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:04.446062 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f45fb4e-563c-4424-9097-3ca0be993882 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:04.448644 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:54:04.454058 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f45fb4e-563c-4424-9097-3ca0be993882 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:54:04.455481 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f45fb4e-563c-4424-9097-3ca0be993882 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:54:04.467996 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-8f45fb4e-563c-4424-9097-3ca0be993882 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 06 21:54:04.467996 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:54:04.467996 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:54:04.467996 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:54:04.467996 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:54:04.467996 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:54:04.467996 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:54:04.467996 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:54:04.467996 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:54:04.467996 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:54:04.467996 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:54:04.467996 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:54:04.467996 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 21:54:04.467996 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 21:54:04.467996 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 21:54:04.467996 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 21:54:04.469064 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:54:04.469064 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:54:04.469064 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 21:54:04.469064 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 21:54:04.469064 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 21:54:04.469064 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 21:54:04.469064 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:54:04.469064 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:54:04.469064 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:54:04.469064 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:54:04.469064 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:54:04.469064 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:54:04.469064 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:54:04.469064 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:54:04.469064 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:54:04.469064 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:54:04.469064 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:54:04.469064 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:54:04.470144 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:54:04.470144 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:54:04.470144 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 21:54:04.470144 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 21:54:04.470144 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 21:54:04.470144 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 21:54:04.470144 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 21:54:04.470144 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 21:54:04.470144 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 06 21:54:04.470144 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:54:04.470144 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 621/1239] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:54:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:54:04.475652 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d996494-9f51-4327-8d23-b8c53e6d18b4 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:54:04.582739 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d996494-9f51-4327-8d23-b8c53e6d18b4 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:54:04.617202 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343eb5a4-14d4-435d-bd8e-2c623e0e07f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:54:04.617768 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343eb5a4-14d4-435d-bd8e-2c623e0e07f8 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:04.618047 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343eb5a4-14d4-435d-bd8e-2c623e0e07f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:04.618318 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343eb5a4-14d4-435d-bd8e-2c623e0e07f8 None admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:04.619254 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343eb5a4-14d4-435d-bd8e-2c623e0e07f8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:54:04.619377 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343eb5a4-14d4-435d-bd8e-2c623e0e07f8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:54:04.620762 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343eb5a4-14d4-435d-bd8e-2c623e0e07f8 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:54:04.638499 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 619/1240] 10.222.0.18 () {56 vars in 1143 bytes} [Sat Aug 6 21:54:04 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:54:04.644256 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f45fb4e-563c-4424-9097-3ca0be993882 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:54:04.744156 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f45fb4e-563c-4424-9097-3ca0be993882 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:54:04.778180 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf754ecd-68ab-4c99-99a0-91525ec93c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:54:04.778775 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf754ecd-68ab-4c99-99a0-91525ec93c52 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:04.779054 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf754ecd-68ab-4c99-99a0-91525ec93c52 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:04.779326 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf754ecd-68ab-4c99-99a0-91525ec93c52 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:04.838595 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:54:04.844932 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf754ecd-68ab-4c99-99a0-91525ec93c52 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:54:04.846417 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf754ecd-68ab-4c99-99a0-91525ec93c52 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:54:04.943874 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 622/1241] 10.222.0.18 () {58 vars in 1282 bytes} [Sat Aug 6 21:54:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 21:54:04.949843 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343eb5a4-14d4-435d-bd8e-2c623e0e07f8 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:54:04.976737 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:54:05.048936 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343eb5a4-14d4-435d-bd8e-2c623e0e07f8 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:54:05.084313 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8c019b-5a33-4983-9ee3-db2bc27ee6c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:54:05.085240 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8c019b-5a33-4983-9ee3-db2bc27ee6c0 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:05.085518 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8c019b-5a33-4983-9ee3-db2bc27ee6c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:05.085789 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8c019b-5a33-4983-9ee3-db2bc27ee6c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:05.095662 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8c019b-5a33-4983-9ee3-db2bc27ee6c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:54:05.097090 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8c019b-5a33-4983-9ee3-db2bc27ee6c0 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:54:05.112179 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-9d8c019b-5a33-4983-9ee3-db2bc27ee6c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:54:05.112179 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:54:05.112179 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:54:05.112179 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:54:05.112179 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:54:05.112179 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:54:05.112179 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:54:05.112179 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:54:05.112179 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:54:05.112179 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:54:05.112179 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:54:05.112179 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:54:05.112179 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:54:05.112179 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:54:05.112179 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:54:05.112179 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:54:05.113525 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:54:05.113525 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:54:05.113525 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:54:05.113525 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:54:05.113525 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:54:05.113525 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:54:05.113525 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:54:05.113525 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:54:05.113525 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:54:05.113525 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:54:05.113525 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:54:05.113525 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:54:05.113525 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:54:05.113525 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:54:05.113525 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:54:05.113525 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:54:05.113525 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:54:05.114739 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:54:05.114739 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:54:05.114739 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:54:05.114739 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 620/1242] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:54:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:54:05.119342 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf754ecd-68ab-4c99-99a0-91525ec93c52 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:54:05.212091 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:54:05.217709 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf754ecd-68ab-4c99-99a0-91525ec93c52 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:54:05.250452 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea329959-a6c2-49f2-9d16-68ae0c7638e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:54:05.251085 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea329959-a6c2-49f2-9d16-68ae0c7638e0 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:05.251360 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea329959-a6c2-49f2-9d16-68ae0c7638e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:05.251626 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea329959-a6c2-49f2-9d16-68ae0c7638e0 None admin] PATH_INFO: `/v3/domains` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:05.252262 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea329959-a6c2-49f2-9d16-68ae0c7638e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:54:05.252587 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea329959-a6c2-49f2-9d16-68ae0c7638e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:54:05.253940 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea329959-a6c2-49f2-9d16-68ae0c7638e0 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:54:05.265059 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 623/1243] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:54:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:54:05.270779 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8c019b-5a33-4983-9ee3-db2bc27ee6c0 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:54:05.352723 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:54:05.373391 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8c019b-5a33-4983-9ee3-db2bc27ee6c0 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:54:05.405693 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf738ec8-cafc-4d7d-97b3-850ca1fa3482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:54:05.406304 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf738ec8-cafc-4d7d-97b3-850ca1fa3482 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:05.406578 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf738ec8-cafc-4d7d-97b3-850ca1fa3482 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:05.406845 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf738ec8-cafc-4d7d-97b3-850ca1fa3482 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:05.413786 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf738ec8-cafc-4d7d-97b3-850ca1fa3482 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:54:05.415213 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf738ec8-cafc-4d7d-97b3-850ca1fa3482 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:54:05.425822 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-cf738ec8-cafc-4d7d-97b3-850ca1fa3482 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 21:54:05.425822 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:54:05.425822 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:54:05.425822 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:54:05.425822 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:54:05.425822 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:54:05.425822 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:54:05.425822 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:54:05.425822 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:54:05.425822 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:54:05.425822 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:54:05.425822 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:54:05.425822 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 21:54:05.425822 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 21:54:05.425822 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 21:54:05.428497 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 21:54:05.428497 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:54:05.428497 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:54:05.428497 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:54:05.428497 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:54:05.428497 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:54:05.428497 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:54:05.428497 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:54:05.428497 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:54:05.428497 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:54:05.428497 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:54:05.428497 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:54:05.428497 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:54:05.428497 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:54:05.428497 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:54:05.428497 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:54:05.428497 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:54:05.429491 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 21:54:05.429491 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 21:54:05.429491 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 21:54:05.429491 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 21:54:05.429491 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 21:54:05.429491 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 21:54:05.429491 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 21:54:05.429491 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:54:05.429491 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 621/1244] 10.222.0.18 () {56 vars in 1111 bytes} [Sat Aug 6 21:54:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:54:05.432914 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea329959-a6c2-49f2-9d16-68ae0c7638e0 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:54:05.535185 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea329959-a6c2-49f2-9d16-68ae0c7638e0 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:54:05.569602 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6beb701-d63f-4c20-bda4-9f04f1b85494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:54:05.570195 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6beb701-d63f-4c20-bda4-9f04f1b85494 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:05.570470 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6beb701-d63f-4c20-bda4-9f04f1b85494 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:05.570737 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6beb701-d63f-4c20-bda4-9f04f1b85494 None admin] PATH_INFO: `/v3/projects` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:05.571434 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6beb701-d63f-4c20-bda4-9f04f1b85494 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:54:05.571754 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6beb701-d63f-4c20-bda4-9f04f1b85494 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:54:05.573179 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6beb701-d63f-4c20-bda4-9f04f1b85494 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:54:05.584995 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 624/1245] 10.222.0.18 () {56 vars in 1148 bytes} [Sat Aug 6 21:54:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:54:05.591143 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf738ec8-cafc-4d7d-97b3-850ca1fa3482 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:54:05.689112 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf738ec8-cafc-4d7d-97b3-850ca1fa3482 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:54:05.720839 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd8cb12-f7cc-40d2-9fe9-9b6f19fe993a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:54:05.721423 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd8cb12-f7cc-40d2-9fe9-9b6f19fe993a None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:05.721697 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd8cb12-f7cc-40d2-9fe9-9b6f19fe993a None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:05.721962 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd8cb12-f7cc-40d2-9fe9-9b6f19fe993a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:05.722665 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd8cb12-f7cc-40d2-9fe9-9b6f19fe993a None admin] RBAC: Adding query filter params (role.id=d90649d4ef604eeb8edcf0aa22932ca1, scope.project.id=4bb6307eb0644d85a1d1daaa61352697, user.id=680d62a67ae2456f8c129e26728c3297) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:54:05.722966 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd8cb12-f7cc-40d2-9fe9-9b6f19fe993a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:54:05.724953 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd8cb12-f7cc-40d2-9fe9-9b6f19fe993a None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:54:05.730584 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 622/1246] 10.222.0.18 () {56 vars in 1374 bytes} [Sat Aug 6 21:54:05 2022] GET /identity/v3/role_assignments?user.id=680d62a67ae2456f8c129e26728c3297&scope.project.id=4bb6307eb0644d85a1d1daaa61352697&role.id=d90649d4ef604eeb8edcf0aa22932ca1 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:54:07.799994 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f7b892-e2f0-42fa-9288-6f279def3673 None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:07.800600 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f7b892-e2f0-42fa-9288-6f279def3673 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:07.800600 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f7b892-e2f0-42fa-9288-6f279def3673 None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:07.801949 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 625/1247] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:54:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:54:07.807339 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bb077f-ce8c-4a48-8495-19c2e881102d None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:07.811434 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bb077f-ce8c-4a48-8495-19c2e881102d None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:07.811434 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bb077f-ce8c-4a48-8495-19c2e881102d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:07.814312 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:54:07.826687 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-c5bb077f-ce8c-4a48-8495-19c2e881102d None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:54:07.827861 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-c5bb077f-ce8c-4a48-8495-19c2e881102d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:54:07.893746 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 623/1248] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:54:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 21:54:07.918359 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f7b892-e2f0-42fa-9288-6f279def3673 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:54:07.937160 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:54:08.017900 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f7b892-e2f0-42fa-9288-6f279def3673 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:54:08.050110 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc74638-5b06-4e65-8824-d2ccf132959b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:54:08.050664 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc74638-5b06-4e65-8824-d2ccf132959b None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:08.050942 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc74638-5b06-4e65-8824-d2ccf132959b None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:08.051218 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc74638-5b06-4e65-8824-d2ccf132959b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:08.061837 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc74638-5b06-4e65-8824-d2ccf132959b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:54:08.066577 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-bbc74638-5b06-4e65-8824-d2ccf132959b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 21:54:08.066577 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:54:08.066577 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:54:08.066577 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:54:08.066577 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:54:08.066577 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:54:08.066577 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:54:08.066577 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:54:08.066577 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:54:08.066577 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:54:08.066577 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:54:08.066577 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:54:08.066577 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:54:08.066577 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:54:08.066577 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:54:08.066577 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:54:08.067674 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:54:08.067674 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:54:08.067674 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:54:08.067674 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:54:08.067674 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:54:08.067674 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:54:08.067674 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:54:08.067674 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:54:08.067674 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:54:08.067674 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:54:08.067674 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:54:08.067674 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:54:08.067674 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:54:08.067674 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:54:08.067674 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:54:08.067674 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:54:08.068827 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:54:08.068827 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:54:08.068827 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:54:08.068827 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:54:08.068827 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:54:08.068827 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:54:08.068827 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:54:08.068827 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:54:08.068827 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:54:08.068827 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:54:08.068827 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 21:54:08.068827 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:54:08.068827 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 626/1249] 10.222.0.18 () {56 vars in 1096 bytes} [Sat Aug 6 21:54:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:54:08.073700 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bb077f-ce8c-4a48-8495-19c2e881102d None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:54:08.170411 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bb077f-ce8c-4a48-8495-19c2e881102d None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:54:08.200127 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:54:08.204765 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afca731-b755-4da5-a39f-92a88ff6fcc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:54:08.205360 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afca731-b755-4da5-a39f-92a88ff6fcc0 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:08.205664 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afca731-b755-4da5-a39f-92a88ff6fcc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:08.205993 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afca731-b755-4da5-a39f-92a88ff6fcc0 None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:08.206627 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afca731-b755-4da5-a39f-92a88ff6fcc0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:54:08.206920 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afca731-b755-4da5-a39f-92a88ff6fcc0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:54:08.208277 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afca731-b755-4da5-a39f-92a88ff6fcc0 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:54:08.216807 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 624/1250] 10.222.0.18 () {56 vars in 1099 bytes} [Sat Aug 6 21:54:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:54:08.222816 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc74638-5b06-4e65-8824-d2ccf132959b None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:54:08.300864 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:54:08.322825 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc74638-5b06-4e65-8824-d2ccf132959b None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:54:08.357139 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6bd670-63f4-49d4-8942-c67b4fa885c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:54:08.357767 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6bd670-63f4-49d4-8942-c67b4fa885c2 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:08.358044 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6bd670-63f4-49d4-8942-c67b4fa885c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:08.358313 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6bd670-63f4-49d4-8942-c67b4fa885c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:08.369012 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6bd670-63f4-49d4-8942-c67b4fa885c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:54:08.370456 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6bd670-63f4-49d4-8942-c67b4fa885c2 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:54:08.384786 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-fd6bd670-63f4-49d4-8942-c67b4fa885c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:54:08.384786 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:54:08.384786 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:54:08.384786 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:54:08.384786 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:54:08.384786 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:54:08.384786 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:54:08.384786 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:54:08.384786 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:54:08.384786 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:54:08.384786 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:54:08.384786 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:54:08.384786 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:54:08.384786 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:54:08.384786 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:54:08.384786 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:54:08.385934 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:54:08.385934 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:54:08.385934 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:54:08.385934 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:54:08.385934 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:54:08.385934 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:54:08.385934 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:54:08.385934 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:54:08.385934 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:54:08.385934 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:54:08.385934 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:54:08.385934 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:54:08.385934 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:54:08.385934 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:54:08.385934 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:54:08.385934 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:54:08.385934 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:54:08.386838 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:54:08.386838 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:54:08.386838 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:54:08.386838 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 627/1251] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:54:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:54:08.391774 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afca731-b755-4da5-a39f-92a88ff6fcc0 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:54:08.491993 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afca731-b755-4da5-a39f-92a88ff6fcc0 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:54:08.524465 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519e7815-4613-4b59-9e0d-4a5ba692fc84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:54:08.525063 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e7815-4613-4b59-9e0d-4a5ba692fc84 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:08.525337 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e7815-4613-4b59-9e0d-4a5ba692fc84 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:08.525602 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e7815-4613-4b59-9e0d-4a5ba692fc84 None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:08.526233 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519e7815-4613-4b59-9e0d-4a5ba692fc84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:54:08.526528 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519e7815-4613-4b59-9e0d-4a5ba692fc84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:54:08.527886 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519e7815-4613-4b59-9e0d-4a5ba692fc84 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:54:08.540147 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 625/1252] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:54:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:54:08.546052 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6bd670-63f4-49d4-8942-c67b4fa885c2 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:54:08.646830 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6bd670-63f4-49d4-8942-c67b4fa885c2 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:54:08.672594 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:54:08.681012 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0c6435-3b6b-4c0e-9cb6-ed826c52fe2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:54:08.681637 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0c6435-3b6b-4c0e-9cb6-ed826c52fe2c None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:08.681938 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0c6435-3b6b-4c0e-9cb6-ed826c52fe2c None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:08.682210 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0c6435-3b6b-4c0e-9cb6-ed826c52fe2c None admin] PATH_INFO: `/v3/users/placement` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:08.689588 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0c6435-3b6b-4c0e-9cb6-ed826c52fe2c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:54:08.691096 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0c6435-3b6b-4c0e-9cb6-ed826c52fe2c None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:54:08.704514 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-2c0c6435-3b6b-4c0e-9cb6-ed826c52fe2c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 06 21:54:08.704514 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:54:08.704514 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:54:08.704514 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:54:08.704514 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:54:08.704514 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:54:08.704514 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:54:08.704514 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:54:08.704514 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:54:08.704514 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:54:08.704514 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:54:08.704514 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:54:08.704514 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 21:54:08.704514 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 21:54:08.704514 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 21:54:08.704514 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 21:54:08.705697 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:54:08.705697 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:54:08.705697 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 21:54:08.705697 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 21:54:08.705697 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 21:54:08.705697 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 21:54:08.705697 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:54:08.705697 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:54:08.705697 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:54:08.705697 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:54:08.705697 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:54:08.705697 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:54:08.705697 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:54:08.705697 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:54:08.705697 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:54:08.705697 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:54:08.705697 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:54:08.705697 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:54:08.706653 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:54:08.706653 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:54:08.706653 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 21:54:08.706653 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 21:54:08.706653 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 21:54:08.706653 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 21:54:08.706653 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 21:54:08.706653 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 21:54:08.706653 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 06 21:54:08.706653 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:54:08.706653 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 628/1253] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:54:08 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:54:08.711895 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519e7815-4613-4b59-9e0d-4a5ba692fc84 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:54:08.766953 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:54:08.810846 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519e7815-4613-4b59-9e0d-4a5ba692fc84 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:54:08.843794 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df5c81f-cd31-4e47-9ab4-2fe43efa88d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:54:08.844374 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df5c81f-cd31-4e47-9ab4-2fe43efa88d4 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:08.844673 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df5c81f-cd31-4e47-9ab4-2fe43efa88d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:08.844936 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df5c81f-cd31-4e47-9ab4-2fe43efa88d4 None admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:08.845641 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df5c81f-cd31-4e47-9ab4-2fe43efa88d4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:54:08.845935 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df5c81f-cd31-4e47-9ab4-2fe43efa88d4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:54:08.847341 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df5c81f-cd31-4e47-9ab4-2fe43efa88d4 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:54:08.864829 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 626/1254] 10.222.0.18 () {56 vars in 1143 bytes} [Sat Aug 6 21:54:08 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:54:08.870713 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0c6435-3b6b-4c0e-9cb6-ed826c52fe2c None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:54:08.971161 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0c6435-3b6b-4c0e-9cb6-ed826c52fe2c None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:54:09.004412 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3782c50a-d0a8-40c4-b351-ad9fb59c750e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:54:09.005019 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3782c50a-d0a8-40c4-b351-ad9fb59c750e None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:09.005295 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3782c50a-d0a8-40c4-b351-ad9fb59c750e None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:09.005559 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3782c50a-d0a8-40c4-b351-ad9fb59c750e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:09.058863 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:54:09.075026 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3782c50a-d0a8-40c4-b351-ad9fb59c750e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:54:09.076504 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3782c50a-d0a8-40c4-b351-ad9fb59c750e None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:54:09.179929 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 629/1255] 10.222.0.18 () {58 vars in 1282 bytes} [Sat Aug 6 21:54:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 21:54:09.185890 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df5c81f-cd31-4e47-9ab4-2fe43efa88d4 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:54:09.291871 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df5c81f-cd31-4e47-9ab4-2fe43efa88d4 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:54:09.305512 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:54:09.326962 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78caff8c-9cb9-4bf4-94e5-140700c3767e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:54:09.327592 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78caff8c-9cb9-4bf4-94e5-140700c3767e None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:09.327895 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78caff8c-9cb9-4bf4-94e5-140700c3767e None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:09.328168 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78caff8c-9cb9-4bf4-94e5-140700c3767e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:09.339059 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78caff8c-9cb9-4bf4-94e5-140700c3767e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:54:09.340553 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78caff8c-9cb9-4bf4-94e5-140700c3767e None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:54:09.355575 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-78caff8c-9cb9-4bf4-94e5-140700c3767e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:54:09.355575 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:54:09.355575 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:54:09.355575 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:54:09.355575 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:54:09.355575 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:54:09.355575 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:54:09.355575 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:54:09.355575 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:54:09.355575 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:54:09.355575 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:54:09.355575 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:54:09.355575 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:54:09.355575 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:54:09.355575 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:54:09.355575 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:54:09.357021 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:54:09.357021 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:54:09.357021 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:54:09.357021 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:54:09.357021 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:54:09.357021 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:54:09.357021 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:54:09.357021 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:54:09.357021 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:54:09.357021 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:54:09.357021 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:54:09.357021 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:54:09.357021 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:54:09.357021 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:54:09.357021 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:54:09.357021 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:54:09.357021 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:54:09.357934 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:54:09.357934 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:54:09.357934 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:54:09.357934 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 627/1256] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:54:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:54:09.362919 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3782c50a-d0a8-40c4-b351-ad9fb59c750e None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:54:09.449217 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:54:09.461414 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3782c50a-d0a8-40c4-b351-ad9fb59c750e None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:54:09.494337 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c3b755-1e2a-4b2d-b437-4e3026797cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:54:09.494934 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c3b755-1e2a-4b2d-b437-4e3026797cbf None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:09.495237 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c3b755-1e2a-4b2d-b437-4e3026797cbf None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:09.495512 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c3b755-1e2a-4b2d-b437-4e3026797cbf None admin] PATH_INFO: `/v3/domains` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:09.496154 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c3b755-1e2a-4b2d-b437-4e3026797cbf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:54:09.496484 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c3b755-1e2a-4b2d-b437-4e3026797cbf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:54:09.497869 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c3b755-1e2a-4b2d-b437-4e3026797cbf None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:54:09.509559 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 630/1257] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:54:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:54:09.515493 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78caff8c-9cb9-4bf4-94e5-140700c3767e None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:54:09.615130 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78caff8c-9cb9-4bf4-94e5-140700c3767e None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:54:09.647549 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c1b1c4-c9f0-4bbc-a440-fd948b81eeb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:54:09.648156 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c1b1c4-c9f0-4bbc-a440-fd948b81eeb3 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:09.648453 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c1b1c4-c9f0-4bbc-a440-fd948b81eeb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:09.648727 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c1b1c4-c9f0-4bbc-a440-fd948b81eeb3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:09.655344 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c1b1c4-c9f0-4bbc-a440-fd948b81eeb3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:54:09.656872 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c1b1c4-c9f0-4bbc-a440-fd948b81eeb3 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:54:09.669068 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-20c1b1c4-c9f0-4bbc-a440-fd948b81eeb3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 21:54:09.669068 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:54:09.669068 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:54:09.669068 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:54:09.669068 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:54:09.669068 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:54:09.669068 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:54:09.669068 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:54:09.669068 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:54:09.669068 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:54:09.669068 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:54:09.669068 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:54:09.669068 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 21:54:09.669068 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 21:54:09.669068 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 21:54:09.670132 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 21:54:09.670132 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:54:09.670132 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:54:09.670132 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:54:09.670132 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:54:09.670132 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:54:09.670132 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:54:09.670132 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:54:09.670132 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:54:09.670132 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:54:09.670132 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:54:09.670132 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:54:09.670132 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:54:09.670132 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:54:09.670132 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:54:09.670132 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:54:09.670132 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:54:09.671172 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 21:54:09.671172 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 21:54:09.671172 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 21:54:09.671172 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 21:54:09.671172 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 21:54:09.671172 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 21:54:09.671172 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 21:54:09.671172 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:54:09.671172 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 628/1258] 10.222.0.18 () {56 vars in 1111 bytes} [Sat Aug 6 21:54:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:54:09.676437 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c3b755-1e2a-4b2d-b437-4e3026797cbf None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:54:09.775757 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c3b755-1e2a-4b2d-b437-4e3026797cbf None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:54:09.808465 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83cbb38-58a8-4635-86af-10363ae9466b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:54:09.809103 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83cbb38-58a8-4635-86af-10363ae9466b None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:09.809384 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83cbb38-58a8-4635-86af-10363ae9466b None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:09.809658 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83cbb38-58a8-4635-86af-10363ae9466b None admin] PATH_INFO: `/v3/projects` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:09.810349 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83cbb38-58a8-4635-86af-10363ae9466b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:54:09.810655 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83cbb38-58a8-4635-86af-10363ae9466b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:54:09.812150 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83cbb38-58a8-4635-86af-10363ae9466b None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:54:09.825541 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 631/1259] 10.222.0.18 () {56 vars in 1148 bytes} [Sat Aug 6 21:54:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:54:09.831604 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c1b1c4-c9f0-4bbc-a440-fd948b81eeb3 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:54:09.857140 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:54:09.928991 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c1b1c4-c9f0-4bbc-a440-fd948b81eeb3 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:54:09.961249 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c068a4d-3417-40c3-bd34-3aca90d11b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:54:09.961834 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c068a4d-3417-40c3-bd34-3aca90d11b7d None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:09.962108 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c068a4d-3417-40c3-bd34-3aca90d11b7d None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:09.962374 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c068a4d-3417-40c3-bd34-3aca90d11b7d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:09.963081 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c068a4d-3417-40c3-bd34-3aca90d11b7d None admin] RBAC: Adding query filter params (role.id=4b8dcf0491fd4072acc69fd894bf77d0, scope.project.id=4bb6307eb0644d85a1d1daaa61352697, user.id=680d62a67ae2456f8c129e26728c3297) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:54:09.963384 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c068a4d-3417-40c3-bd34-3aca90d11b7d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:54:09.964825 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c068a4d-3417-40c3-bd34-3aca90d11b7d None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:54:09.969930 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 629/1260] 10.222.0.18 () {56 vars in 1374 bytes} [Sat Aug 6 21:54:09 2022] GET /identity/v3/role_assignments?user.id=680d62a67ae2456f8c129e26728c3297&scope.project.id=4bb6307eb0644d85a1d1daaa61352697&role.id=4b8dcf0491fd4072acc69fd894bf77d0 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:54:11.918421 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a53828-775f-4048-95e8-dbdbc855735d None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:11.918996 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a53828-775f-4048-95e8-dbdbc855735d None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:11.918996 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a53828-775f-4048-95e8-dbdbc855735d None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:11.920479 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 632/1261] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:54:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:54:11.925415 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6cb5d0-b18c-4c1b-93c7-5d636216db82 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:11.925802 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6cb5d0-b18c-4c1b-93c7-5d636216db82 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:11.925958 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6cb5d0-b18c-4c1b-93c7-5d636216db82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:11.944592 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-0c6cb5d0-b18c-4c1b-93c7-5d636216db82 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:54:11.945759 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-0c6cb5d0-b18c-4c1b-93c7-5d636216db82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:54:12.009808 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 630/1262] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:54:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 21:54:12.031715 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a53828-775f-4048-95e8-dbdbc855735d None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:54:12.042584 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:54:12.131848 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a53828-775f-4048-95e8-dbdbc855735d None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:54:12.164833 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d050d9d-a9fc-483e-820a-657027b1c5d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:54:12.165448 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d050d9d-a9fc-483e-820a-657027b1c5d0 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:12.165722 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d050d9d-a9fc-483e-820a-657027b1c5d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:12.165989 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d050d9d-a9fc-483e-820a-657027b1c5d0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:12.175823 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d050d9d-a9fc-483e-820a-657027b1c5d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:54:12.180997 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-6d050d9d-a9fc-483e-820a-657027b1c5d0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 21:54:12.180997 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:54:12.180997 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:54:12.180997 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:54:12.180997 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:54:12.180997 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:54:12.180997 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:54:12.180997 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:54:12.180997 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:54:12.180997 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:54:12.180997 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:54:12.180997 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:54:12.180997 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:54:12.180997 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:54:12.180997 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:54:12.180997 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:54:12.182142 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:54:12.182142 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:54:12.182142 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:54:12.182142 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:54:12.182142 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:54:12.182142 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:54:12.182142 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:54:12.182142 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:54:12.182142 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:54:12.182142 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:54:12.182142 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:54:12.182142 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:54:12.182142 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:54:12.182142 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:54:12.182142 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:54:12.182142 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:54:12.183390 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:54:12.183390 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:54:12.183390 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:54:12.183390 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:54:12.183390 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:54:12.183390 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:54:12.183390 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:54:12.183390 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:54:12.183390 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:54:12.183390 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:54:12.183390 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 21:54:12.183390 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:54:12.183390 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 633/1263] 10.222.0.18 () {56 vars in 1096 bytes} [Sat Aug 6 21:54:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:54:12.188193 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6cb5d0-b18c-4c1b-93c7-5d636216db82 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:54:12.206741 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:54:12.285552 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6cb5d0-b18c-4c1b-93c7-5d636216db82 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:54:12.320904 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cece0755-0497-43fe-870e-c63785a37203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:54:12.321565 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cece0755-0497-43fe-870e-c63785a37203 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:12.321843 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cece0755-0497-43fe-870e-c63785a37203 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:12.322113 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cece0755-0497-43fe-870e-c63785a37203 None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:12.322769 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cece0755-0497-43fe-870e-c63785a37203 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:54:12.323069 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cece0755-0497-43fe-870e-c63785a37203 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:54:12.324453 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cece0755-0497-43fe-870e-c63785a37203 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:54:12.332313 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 631/1264] 10.222.0.18 () {56 vars in 1099 bytes} [Sat Aug 6 21:54:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:54:12.338101 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d050d9d-a9fc-483e-820a-657027b1c5d0 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:54:12.405276 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:54:12.438525 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d050d9d-a9fc-483e-820a-657027b1c5d0 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:54:12.470382 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105338f3-5b0b-4627-9544-21bdf193432b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:54:12.471000 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105338f3-5b0b-4627-9544-21bdf193432b None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:12.471278 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105338f3-5b0b-4627-9544-21bdf193432b None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:12.471549 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105338f3-5b0b-4627-9544-21bdf193432b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:12.482415 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105338f3-5b0b-4627-9544-21bdf193432b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:54:12.483847 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105338f3-5b0b-4627-9544-21bdf193432b None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:54:12.498660 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-105338f3-5b0b-4627-9544-21bdf193432b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:54:12.498660 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:54:12.498660 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:54:12.498660 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:54:12.498660 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:54:12.498660 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:54:12.498660 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:54:12.498660 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:54:12.498660 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:54:12.498660 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:54:12.498660 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:54:12.498660 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:54:12.498660 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:54:12.498660 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:54:12.498660 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:54:12.498660 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:54:12.499802 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:54:12.499802 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:54:12.499802 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:54:12.499802 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:54:12.499802 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:54:12.499802 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:54:12.499802 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:54:12.499802 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:54:12.499802 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:54:12.499802 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:54:12.499802 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:54:12.499802 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:54:12.499802 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:54:12.499802 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:54:12.499802 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:54:12.499802 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:54:12.499802 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:54:12.501013 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:54:12.501013 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:54:12.501013 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:54:12.501013 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 634/1265] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:54:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:54:12.505545 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cece0755-0497-43fe-870e-c63785a37203 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:54:12.586805 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:54:12.605727 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cece0755-0497-43fe-870e-c63785a37203 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:54:12.638396 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed98563-c7a9-4ab4-8a47-f4c415c65de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:54:12.641168 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed98563-c7a9-4ab4-8a47-f4c415c65de7 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:12.641168 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed98563-c7a9-4ab4-8a47-f4c415c65de7 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:12.641168 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed98563-c7a9-4ab4-8a47-f4c415c65de7 None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:12.641168 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed98563-c7a9-4ab4-8a47-f4c415c65de7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:54:12.641168 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed98563-c7a9-4ab4-8a47-f4c415c65de7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:54:12.641780 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed98563-c7a9-4ab4-8a47-f4c415c65de7 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:54:12.653475 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 632/1266] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:54:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:54:12.659161 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105338f3-5b0b-4627-9544-21bdf193432b None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:54:12.757689 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105338f3-5b0b-4627-9544-21bdf193432b None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:54:12.776952 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:54:12.790878 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcdff18-d428-4706-8b27-53233d713100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:54:12.791490 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcdff18-d428-4706-8b27-53233d713100 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:12.791765 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcdff18-d428-4706-8b27-53233d713100 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:12.792034 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcdff18-d428-4706-8b27-53233d713100 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:12.799161 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcdff18-d428-4706-8b27-53233d713100 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:54:12.800628 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcdff18-d428-4706-8b27-53233d713100 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:54:12.811822 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-2dcdff18-d428-4706-8b27-53233d713100 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 06 21:54:12.811822 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:54:12.811822 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:54:12.811822 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:54:12.811822 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:54:12.811822 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:54:12.811822 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:54:12.811822 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:54:12.811822 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:54:12.811822 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:54:12.811822 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:54:12.811822 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:54:12.811822 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 21:54:12.811822 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 21:54:12.811822 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 21:54:12.811822 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 21:54:12.812968 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:54:12.812968 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:54:12.812968 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 21:54:12.812968 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 21:54:12.812968 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 21:54:12.812968 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 21:54:12.812968 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:54:12.812968 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:54:12.812968 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:54:12.812968 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:54:12.812968 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:54:12.812968 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:54:12.812968 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:54:12.812968 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:54:12.812968 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:54:12.812968 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:54:12.812968 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:54:12.812968 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:54:12.813954 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:54:12.813954 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:54:12.813954 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 21:54:12.813954 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 21:54:12.813954 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 21:54:12.813954 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 21:54:12.813954 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 21:54:12.813954 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 21:54:12.813954 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 06 21:54:12.813954 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:54:12.813954 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 635/1267] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:54:12 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:54:12.818930 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed98563-c7a9-4ab4-8a47-f4c415c65de7 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:54:12.916921 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed98563-c7a9-4ab4-8a47-f4c415c65de7 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:54:12.950462 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0136c3f-ec56-46a7-b671-a9e23f83250c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:54:12.951044 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0136c3f-ec56-46a7-b671-a9e23f83250c None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:12.951317 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0136c3f-ec56-46a7-b671-a9e23f83250c None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:12.951589 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0136c3f-ec56-46a7-b671-a9e23f83250c None admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:12.952338 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0136c3f-ec56-46a7-b671-a9e23f83250c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:54:12.952673 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0136c3f-ec56-46a7-b671-a9e23f83250c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:54:12.954088 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0136c3f-ec56-46a7-b671-a9e23f83250c None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:54:12.963155 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:54:12.972003 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 633/1268] 10.222.0.18 () {56 vars in 1143 bytes} [Sat Aug 6 21:54:12 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:54:12.977708 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcdff18-d428-4706-8b27-53233d713100 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:54:13.075760 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcdff18-d428-4706-8b27-53233d713100 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:54:13.109434 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a6f9e8-5a14-44c2-b35f-7189233ff1ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:54:13.110056 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a6f9e8-5a14-44c2-b35f-7189233ff1ef None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:13.110340 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a6f9e8-5a14-44c2-b35f-7189233ff1ef None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:13.110614 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a6f9e8-5a14-44c2-b35f-7189233ff1ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:13.121711 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a6f9e8-5a14-44c2-b35f-7189233ff1ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:54:13.123148 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a6f9e8-5a14-44c2-b35f-7189233ff1ef None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:54:13.137464 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-33a6f9e8-5a14-44c2-b35f-7189233ff1ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:54:13.137464 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:54:13.137464 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:54:13.137464 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:54:13.137464 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:54:13.137464 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:54:13.137464 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:54:13.137464 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:54:13.137464 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:54:13.137464 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:54:13.137464 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:54:13.137464 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:54:13.137464 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:54:13.137464 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:54:13.137464 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:54:13.137464 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:54:13.138613 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:54:13.138613 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:54:13.138613 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:54:13.138613 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:54:13.138613 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:54:13.138613 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:54:13.138613 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:54:13.138613 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:54:13.138613 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:54:13.138613 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:54:13.138613 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:54:13.138613 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:54:13.138613 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:54:13.138613 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:54:13.138613 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:54:13.138613 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:54:13.138613 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:54:13.139705 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:54:13.139705 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:54:13.139705 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:54:13.139705 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 636/1269] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:54:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:54:13.144315 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0136c3f-ec56-46a7-b671-a9e23f83250c None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:54:13.242896 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0136c3f-ec56-46a7-b671-a9e23f83250c None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:54:13.276920 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76af8be0-c51d-4de9-9f43-7426be123e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:54:13.277517 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76af8be0-c51d-4de9-9f43-7426be123e52 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:13.277797 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76af8be0-c51d-4de9-9f43-7426be123e52 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:13.278069 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76af8be0-c51d-4de9-9f43-7426be123e52 None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:13.278702 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76af8be0-c51d-4de9-9f43-7426be123e52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:54:13.279020 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76af8be0-c51d-4de9-9f43-7426be123e52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:54:13.280435 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76af8be0-c51d-4de9-9f43-7426be123e52 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:54:13.291386 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 634/1270] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:54:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:54:13.297172 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a6f9e8-5a14-44c2-b35f-7189233ff1ef None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:54:13.329910 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:54:13.396679 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a6f9e8-5a14-44c2-b35f-7189233ff1ef None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:54:13.428829 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241f442d-c0d3-4520-b214-d5099c34d42c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:54:13.429447 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241f442d-c0d3-4520-b214-d5099c34d42c None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:13.429721 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241f442d-c0d3-4520-b214-d5099c34d42c None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:13.429987 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241f442d-c0d3-4520-b214-d5099c34d42c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:13.436263 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241f442d-c0d3-4520-b214-d5099c34d42c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:54:13.437709 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241f442d-c0d3-4520-b214-d5099c34d42c None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:54:13.449155 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-241f442d-c0d3-4520-b214-d5099c34d42c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 21:54:13.449155 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:54:13.449155 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:54:13.449155 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:54:13.449155 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:54:13.449155 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:54:13.449155 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:54:13.449155 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:54:13.449155 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:54:13.449155 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:54:13.449155 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:54:13.449155 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:54:13.449155 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 21:54:13.449155 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 21:54:13.449155 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 21:54:13.450225 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 21:54:13.450225 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:54:13.450225 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:54:13.450225 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:54:13.450225 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:54:13.450225 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:54:13.450225 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:54:13.450225 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:54:13.450225 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:54:13.450225 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:54:13.450225 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:54:13.450225 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:54:13.450225 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:54:13.450225 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:54:13.450225 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:54:13.450225 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:54:13.450225 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:54:13.451300 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 21:54:13.451300 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 21:54:13.451300 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 21:54:13.451300 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 21:54:13.451300 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 21:54:13.451300 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 21:54:13.451300 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 21:54:13.451300 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:54:13.451300 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 637/1271] 10.222.0.18 () {56 vars in 1111 bytes} [Sat Aug 6 21:54:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:54:13.456264 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76af8be0-c51d-4de9-9f43-7426be123e52 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:54:13.513807 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:54:13.553370 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76af8be0-c51d-4de9-9f43-7426be123e52 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:54:13.586565 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5f984f-5968-4685-a6c1-f082d0c517dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:54:13.587110 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5f984f-5968-4685-a6c1-f082d0c517dd None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:13.587390 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5f984f-5968-4685-a6c1-f082d0c517dd None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:13.587664 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5f984f-5968-4685-a6c1-f082d0c517dd None admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:13.588390 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5f984f-5968-4685-a6c1-f082d0c517dd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:54:13.588727 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5f984f-5968-4685-a6c1-f082d0c517dd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:54:13.590159 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5f984f-5968-4685-a6c1-f082d0c517dd None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:54:13.602650 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 635/1272] 10.222.0.18 () {56 vars in 1148 bytes} [Sat Aug 6 21:54:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:54:13.608352 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241f442d-c0d3-4520-b214-d5099c34d42c None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:54:13.697687 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:54:13.705842 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241f442d-c0d3-4520-b214-d5099c34d42c None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:54:13.738518 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428ff324-99ac-4f96-a659-f17dab5cc767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:54:13.739161 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428ff324-99ac-4f96-a659-f17dab5cc767 None admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:13.740085 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428ff324-99ac-4f96-a659-f17dab5cc767 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:13.740370 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428ff324-99ac-4f96-a659-f17dab5cc767 None admin] PATH_INFO: `/v3/projects/4bb6307eb0644d85a1d1daaa61352697/users/680d62a67ae2456f8c129e26728c3297/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:13.742290 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428ff324-99ac-4f96-a659-f17dab5cc767 None admin] RBAC: Authorizing `identity:create_grant(project_id=4bb6307eb0644d85a1d1daaa61352697, user_id=680d62a67ae2456f8c129e26728c3297, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:54:13.743707 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428ff324-99ac-4f96-a659-f17dab5cc767 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:54:13.846442 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 638/1273] 10.222.0.18 () {58 vars in 1439 bytes} [Sat Aug 6 21:54:13 2022] PUT /identity/v3/projects/4bb6307eb0644d85a1d1daaa61352697/users/680d62a67ae2456f8c129e26728c3297/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 21:54:15.834709 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636386f9-ceb3-4e91-860b-8b94d2b54305 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:15.835268 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636386f9-ceb3-4e91-860b-8b94d2b54305 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:15.835268 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636386f9-ceb3-4e91-860b-8b94d2b54305 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:15.836598 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 636/1274] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:54:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:54:15.841849 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d8d1c0-27e1-4de9-a8b9-ed4b4d8ba6f4 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:15.842243 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d8d1c0-27e1-4de9-a8b9-ed4b4d8ba6f4 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:15.842399 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d8d1c0-27e1-4de9-a8b9-ed4b4d8ba6f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:15.860163 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-d0d8d1c0-27e1-4de9-a8b9-ed4b4d8ba6f4 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:54:15.861366 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-d0d8d1c0-27e1-4de9-a8b9-ed4b4d8ba6f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:54:15.925141 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 639/1275] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:54:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 21:54:15.948984 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636386f9-ceb3-4e91-860b-8b94d2b54305 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:54:16.048925 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636386f9-ceb3-4e91-860b-8b94d2b54305 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:54:16.066120 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:54:16.081972 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea49180-8146-4e94-a53a-0bd4053600a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:54:16.082587 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea49180-8146-4e94-a53a-0bd4053600a1 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:16.082862 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea49180-8146-4e94-a53a-0bd4053600a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:16.083215 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea49180-8146-4e94-a53a-0bd4053600a1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:16.093077 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea49180-8146-4e94-a53a-0bd4053600a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 21:54:16.097521 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-eea49180-8146-4e94-a53a-0bd4053600a1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 21:54:16.097521 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:54:16.097521 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:54:16.097521 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:54:16.097521 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:54:16.097521 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:54:16.097521 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:54:16.097521 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:54:16.097521 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:54:16.097521 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:54:16.097521 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:54:16.097521 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:54:16.097521 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 21:54:16.097521 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 21:54:16.097521 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 21:54:16.097521 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 21:54:16.099187 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 21:54:16.099187 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 21:54:16.099187 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 21:54:16.099187 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 21:54:16.099187 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:54:16.099187 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:54:16.099187 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:54:16.099187 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:54:16.099187 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:54:16.099187 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:54:16.099187 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:54:16.099187 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:54:16.099187 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:54:16.099187 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:54:16.099187 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:54:16.099187 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:54:16.100168 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:54:16.100168 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:54:16.100168 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:54:16.100168 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:54:16.100168 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 21:54:16.100168 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 21:54:16.100168 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 21:54:16.100168 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 21:54:16.100168 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 21:54:16.100168 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 21:54:16.100168 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 21:54:16.100168 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:54:16.100168 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 637/1276] 10.222.0.18 () {56 vars in 1096 bytes} [Sat Aug 6 21:54:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:54:16.105507 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d8d1c0-27e1-4de9-a8b9-ed4b4d8ba6f4 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:54:16.183998 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:54:16.200872 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d8d1c0-27e1-4de9-a8b9-ed4b4d8ba6f4 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:54:16.233314 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea1799e-ae55-4936-a4e8-0f455213f86a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:54:16.233921 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea1799e-ae55-4936-a4e8-0f455213f86a None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:16.234203 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea1799e-ae55-4936-a4e8-0f455213f86a None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:16.234474 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea1799e-ae55-4936-a4e8-0f455213f86a None admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:16.235137 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea1799e-ae55-4936-a4e8-0f455213f86a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:54:16.235438 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea1799e-ae55-4936-a4e8-0f455213f86a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:54:16.236834 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea1799e-ae55-4936-a4e8-0f455213f86a None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:54:16.245534 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 640/1277] 10.222.0.18 () {56 vars in 1099 bytes} [Sat Aug 6 21:54:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:54:16.251278 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea49180-8146-4e94-a53a-0bd4053600a1 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:54:16.351112 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea49180-8146-4e94-a53a-0bd4053600a1 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:54:16.384877 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23120e65-6adf-40e5-a3b2-58f3904d09f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:54:16.385494 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23120e65-6adf-40e5-a3b2-58f3904d09f2 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:16.385768 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23120e65-6adf-40e5-a3b2-58f3904d09f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:16.386033 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23120e65-6adf-40e5-a3b2-58f3904d09f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:16.396375 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23120e65-6adf-40e5-a3b2-58f3904d09f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:54:16.397842 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23120e65-6adf-40e5-a3b2-58f3904d09f2 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:54:16.413055 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-23120e65-6adf-40e5-a3b2-58f3904d09f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:54:16.413055 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:54:16.413055 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:54:16.413055 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:54:16.413055 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:54:16.413055 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:54:16.413055 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:54:16.413055 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:54:16.413055 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:54:16.413055 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:54:16.413055 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:54:16.413055 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:54:16.413055 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:54:16.413055 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:54:16.413055 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:54:16.413055 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:54:16.414205 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:54:16.414205 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:54:16.414205 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:54:16.414205 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:54:16.414205 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:54:16.414205 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:54:16.414205 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:54:16.414205 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:54:16.414205 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:54:16.414205 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:54:16.414205 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:54:16.414205 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:54:16.414205 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:54:16.414205 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:54:16.414205 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:54:16.414205 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:54:16.414205 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:54:16.415108 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:54:16.415108 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:54:16.415108 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:54:16.415108 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 638/1278] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:54:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 21:54:16.420044 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea1799e-ae55-4936-a4e8-0f455213f86a None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:54:16.519344 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea1799e-ae55-4936-a4e8-0f455213f86a None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:54:16.552089 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cf9df3-6ba4-4d9e-8cf3-85c04bb04088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:54:16.552705 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cf9df3-6ba4-4d9e-8cf3-85c04bb04088 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:16.552979 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cf9df3-6ba4-4d9e-8cf3-85c04bb04088 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:16.553245 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cf9df3-6ba4-4d9e-8cf3-85c04bb04088 None admin] PATH_INFO: `/v3/domains` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:16.553882 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cf9df3-6ba4-4d9e-8cf3-85c04bb04088 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:54:16.554169 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cf9df3-6ba4-4d9e-8cf3-85c04bb04088 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:54:16.555521 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cf9df3-6ba4-4d9e-8cf3-85c04bb04088 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:54:16.566791 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 641/1279] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:54:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:54:16.572946 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23120e65-6adf-40e5-a3b2-58f3904d09f2 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:54:16.605676 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:54:16.673657 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23120e65-6adf-40e5-a3b2-58f3904d09f2 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:54:16.705678 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8335f9-ee18-4868-99f8-02b65f157cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:54:16.706297 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8335f9-ee18-4868-99f8-02b65f157cf9 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:16.706601 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8335f9-ee18-4868-99f8-02b65f157cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:16.706869 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8335f9-ee18-4868-99f8-02b65f157cf9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:16.714635 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8335f9-ee18-4868-99f8-02b65f157cf9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:54:16.716053 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8335f9-ee18-4868-99f8-02b65f157cf9 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:54:16.726687 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-6e8335f9-ee18-4868-99f8-02b65f157cf9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 06 21:54:16.726687 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:54:16.726687 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:54:16.726687 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:54:16.726687 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:54:16.726687 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:54:16.726687 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:54:16.726687 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:54:16.726687 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:54:16.726687 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:54:16.726687 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:54:16.726687 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:54:16.726687 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 21:54:16.726687 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 21:54:16.726687 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 21:54:16.726687 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 21:54:16.732950 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:54:16.732950 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:54:16.732950 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 21:54:16.732950 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 21:54:16.732950 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 21:54:16.732950 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 21:54:16.732950 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:54:16.732950 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:54:16.732950 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:54:16.732950 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:54:16.732950 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:54:16.732950 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:54:16.732950 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:54:16.732950 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:54:16.732950 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:54:16.732950 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:54:16.732950 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:54:16.732950 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:54:16.733934 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:54:16.733934 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:54:16.733934 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 21:54:16.733934 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 21:54:16.733934 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 21:54:16.733934 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 21:54:16.733934 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 21:54:16.733934 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 21:54:16.733934 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 06 21:54:16.733934 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:54:16.733934 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 639/1280] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:54:16 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:54:16.734556 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cf9df3-6ba4-4d9e-8cf3-85c04bb04088 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:54:16.738101 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:54:16.833905 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cf9df3-6ba4-4d9e-8cf3-85c04bb04088 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:54:16.866777 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c937577e-0d5a-46e4-beb5-d0aa462763f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:54:16.867368 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c937577e-0d5a-46e4-beb5-d0aa462763f6 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:16.867606 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c937577e-0d5a-46e4-beb5-d0aa462763f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:16.867879 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c937577e-0d5a-46e4-beb5-d0aa462763f6 None admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:16.868632 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c937577e-0d5a-46e4-beb5-d0aa462763f6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:54:16.868937 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c937577e-0d5a-46e4-beb5-d0aa462763f6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:54:16.870357 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c937577e-0d5a-46e4-beb5-d0aa462763f6 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:54:16.887345 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 642/1281] 10.222.0.18 () {56 vars in 1143 bytes} [Sat Aug 6 21:54:16 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:54:16.893192 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8335f9-ee18-4868-99f8-02b65f157cf9 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:54:16.983251 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:54:16.992793 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8335f9-ee18-4868-99f8-02b65f157cf9 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:54:17.026071 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd50a2c-c103-4f67-bb28-499d8dac0960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:54:17.026668 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd50a2c-c103-4f67-bb28-499d8dac0960 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:17.026947 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd50a2c-c103-4f67-bb28-499d8dac0960 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:17.027220 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd50a2c-c103-4f67-bb28-499d8dac0960 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:17.093479 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd50a2c-c103-4f67-bb28-499d8dac0960 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:54:17.094916 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd50a2c-c103-4f67-bb28-499d8dac0960 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:54:17.180762 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:54:17.190715 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 640/1282] 10.222.0.18 () {58 vars in 1282 bytes} [Sat Aug 6 21:54:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 21:54:17.197104 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c937577e-0d5a-46e4-beb5-d0aa462763f6 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:54:17.257462 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:54:17.293160 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c937577e-0d5a-46e4-beb5-d0aa462763f6 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:54:17.325787 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c27ccab-529b-47d9-b48d-ca05261b7368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:54:17.326399 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c27ccab-529b-47d9-b48d-ca05261b7368 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:17.326673 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c27ccab-529b-47d9-b48d-ca05261b7368 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:17.326942 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c27ccab-529b-47d9-b48d-ca05261b7368 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:17.337002 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c27ccab-529b-47d9-b48d-ca05261b7368 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:54:17.338435 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c27ccab-529b-47d9-b48d-ca05261b7368 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:54:17.352255 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-3c27ccab-529b-47d9-b48d-ca05261b7368 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:54:17.352255 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:54:17.352255 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:54:17.352255 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:54:17.352255 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:54:17.352255 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:54:17.352255 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:54:17.352255 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:54:17.352255 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:54:17.352255 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:54:17.352255 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:54:17.352255 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:54:17.352255 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 21:54:17.352255 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 21:54:17.352255 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 21:54:17.352255 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 21:54:17.353418 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:54:17.353418 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:54:17.353418 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:54:17.353418 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:54:17.353418 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:54:17.353418 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:54:17.353418 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:54:17.353418 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:54:17.353418 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:54:17.353418 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:54:17.353418 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:54:17.353418 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:54:17.353418 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:54:17.353418 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:54:17.353418 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:54:17.353418 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:54:17.353418 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 21:54:17.354538 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 21:54:17.354538 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 21:54:17.354538 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:54:17.354538 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 643/1283] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 21:54:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:54:17.359473 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd50a2c-c103-4f67-bb28-499d8dac0960 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:54:17.458219 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd50a2c-c103-4f67-bb28-499d8dac0960 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:54:17.492190 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6be61dd-43a2-42d9-9c61-47f4271b9420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:54:17.492805 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6be61dd-43a2-42d9-9c61-47f4271b9420 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:17.493081 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6be61dd-43a2-42d9-9c61-47f4271b9420 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:17.493349 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6be61dd-43a2-42d9-9c61-47f4271b9420 None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:17.493976 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6be61dd-43a2-42d9-9c61-47f4271b9420 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:54:17.494275 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6be61dd-43a2-42d9-9c61-47f4271b9420 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:54:17.495629 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6be61dd-43a2-42d9-9c61-47f4271b9420 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:54:17.506736 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 641/1284] 10.222.0.18 () {56 vars in 1109 bytes} [Sat Aug 6 21:54:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:54:17.512448 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c27ccab-529b-47d9-b48d-ca05261b7368 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:54:17.612521 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c27ccab-529b-47d9-b48d-ca05261b7368 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:54:17.626127 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:54:17.645738 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048dc155-568d-4a99-9363-6b0a7aa94cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:54:17.646369 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048dc155-568d-4a99-9363-6b0a7aa94cad None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:17.646639 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048dc155-568d-4a99-9363-6b0a7aa94cad None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:17.646905 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048dc155-568d-4a99-9363-6b0a7aa94cad None admin] PATH_INFO: `/v3/projects/service` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:17.653371 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048dc155-568d-4a99-9363-6b0a7aa94cad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:54:17.654813 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048dc155-568d-4a99-9363-6b0a7aa94cad None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:54:17.667163 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application [None req-048dc155-568d-4a99-9363-6b0a7aa94cad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 21:54:17.667163 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:54:17.667163 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:54:17.667163 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:54:17.667163 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:54:17.667163 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:54:17.667163 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:54:17.667163 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:54:17.667163 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:54:17.667163 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:54:17.667163 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:54:17.667163 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:54:17.667163 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 21:54:17.667163 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 21:54:17.667163 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 21:54:17.668228 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 21:54:17.668228 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:54:17.668228 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:54:17.668228 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:54:17.668228 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:54:17.668228 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:54:17.668228 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:54:17.668228 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:54:17.668228 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application with Lock( Aug 06 21:54:17.668228 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:54:17.668228 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:54:17.668228 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:54:17.668228 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:54:17.668228 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:54:17.668228 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:54:17.668228 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:54:17.668228 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:54:17.669523 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 21:54:17.669523 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 21:54:17.669523 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 21:54:17.669523 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 21:54:17.669523 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 21:54:17.669523 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 21:54:17.669523 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 21:54:17.669523 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: ERROR keystone.server.flask.application Aug 06 21:54:17.669523 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 644/1285] 10.222.0.18 () {56 vars in 1111 bytes} [Sat Aug 6 21:54:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:54:17.674146 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6be61dd-43a2-42d9-9c61-47f4271b9420 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:54:17.722404 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:54:17.771183 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6be61dd-43a2-42d9-9c61-47f4271b9420 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:54:17.805357 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12426092-ac06-4f2b-8580-5c778772ee7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:54:17.805949 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12426092-ac06-4f2b-8580-5c778772ee7d None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:17.806233 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12426092-ac06-4f2b-8580-5c778772ee7d None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:17.806505 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12426092-ac06-4f2b-8580-5c778772ee7d None admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:17.807186 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12426092-ac06-4f2b-8580-5c778772ee7d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:54:17.807486 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12426092-ac06-4f2b-8580-5c778772ee7d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:54:17.808958 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12426092-ac06-4f2b-8580-5c778772ee7d None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:54:17.821542 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 642/1286] 10.222.0.18 () {56 vars in 1148 bytes} [Sat Aug 6 21:54:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:54:17.827661 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048dc155-568d-4a99-9363-6b0a7aa94cad None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:54:17.927916 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048dc155-568d-4a99-9363-6b0a7aa94cad None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:54:17.960624 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8a4eb4-e7d8-481c-90df-ff3e60641493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:54:17.961214 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8a4eb4-e7d8-481c-90df-ff3e60641493 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:17.961492 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8a4eb4-e7d8-481c-90df-ff3e60641493 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:17.961762 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8a4eb4-e7d8-481c-90df-ff3e60641493 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:17.962475 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8a4eb4-e7d8-481c-90df-ff3e60641493 None admin] RBAC: Adding query filter params (role.id=4b8dcf0491fd4072acc69fd894bf77d0, scope.project.id=4bb6307eb0644d85a1d1daaa61352697, user.id=680d62a67ae2456f8c129e26728c3297) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:54:17.962781 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8a4eb4-e7d8-481c-90df-ff3e60641493 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:54:17.964215 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8a4eb4-e7d8-481c-90df-ff3e60641493 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:54:17.969826 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 645/1287] 10.222.0.18 () {56 vars in 1374 bytes} [Sat Aug 6 21:54:17 2022] GET /identity/v3/role_assignments?user.id=680d62a67ae2456f8c129e26728c3297&scope.project.id=4bb6307eb0644d85a1d1daaa61352697&role.id=4b8dcf0491fd4072acc69fd894bf77d0 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:54:19.975199 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9317d388-3ef1-44b4-a09b-e0a33257f9a0 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:19.975727 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9317d388-3ef1-44b4-a09b-e0a33257f9a0 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:19.975727 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9317d388-3ef1-44b4-a09b-e0a33257f9a0 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:19.977087 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 643/1288] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:54:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 21:54:19.982506 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e00fe-4b32-405c-9308-55857b832b3d None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:19.982885 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e00fe-4b32-405c-9308-55857b832b3d None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:19.983059 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e00fe-4b32-405c-9308-55857b832b3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:20.000995 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-6a6e00fe-4b32-405c-9308-55857b832b3d None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:54:20.002170 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-6a6e00fe-4b32-405c-9308-55857b832b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:54:20.032224 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:54:20.070725 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 646/1289] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:54:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 21:54:20.092186 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9317d388-3ef1-44b4-a09b-e0a33257f9a0 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:54:20.193309 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9317d388-3ef1-44b4-a09b-e0a33257f9a0 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:54:20.203529 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:54:20.225962 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a998281c-f38c-4567-856c-90926e24e563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:54:20.226516 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a998281c-f38c-4567-856c-90926e24e563 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:20.226789 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a998281c-f38c-4567-856c-90926e24e563 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:20.227056 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a998281c-f38c-4567-856c-90926e24e563 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:20.227691 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a998281c-f38c-4567-856c-90926e24e563 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:54:20.229071 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a998281c-f38c-4567-856c-90926e24e563 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:54:20.237718 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-a998281c-f38c-4567-856c-90926e24e563 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 06 21:54:20.237718 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:54:20.237718 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 21:54:20.237718 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 21:54:20.237718 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 21:54:20.237718 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 21:54:20.237718 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 21:54:20.237718 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 21:54:20.237718 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 06 21:54:20.237718 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:54:20.237718 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 21:54:20.237718 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:54:20.237718 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:54:20.237718 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:54:20.237718 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:54:20.237718 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:54:20.237718 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:54:20.241280 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:54:20.241280 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:54:20.241280 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:54:20.241280 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:54:20.241280 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:54:20.241280 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:54:20.241280 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 21:54:20.241280 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 21:54:20.241280 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 21:54:20.241280 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 21:54:20.241280 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:54:20.241280 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:54:20.241280 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:54:20.241280 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:54:20.241280 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:54:20.241280 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:54:20.242239 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:54:20.242239 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:54:20.242239 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:54:20.242239 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:54:20.242239 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:54:20.242239 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:54:20.242239 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:54:20.242239 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:54:20.242239 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:54:20.242239 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:54:20.242239 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 21:54:20.242239 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 21:54:20.242239 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 06 21:54:20.242239 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:54:20.242239 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 644/1290] 10.222.0.18 () {56 vars in 1117 bytes} [Sat Aug 6 21:54:20 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:54:20.245519 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6e00fe-4b32-405c-9308-55857b832b3d None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:54:20.340708 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6e00fe-4b32-405c-9308-55857b832b3d None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:54:20.373869 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de76b0a-218d-4a99-916d-e1fa5e42efd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:54:20.374468 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de76b0a-218d-4a99-916d-e1fa5e42efd5 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:20.374761 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de76b0a-218d-4a99-916d-e1fa5e42efd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:20.375029 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de76b0a-218d-4a99-916d-e1fa5e42efd5 None admin] PATH_INFO: `/v3/services` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:20.375648 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de76b0a-218d-4a99-916d-e1fa5e42efd5 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:54:20.375944 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de76b0a-218d-4a99-916d-e1fa5e42efd5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:54:20.377313 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de76b0a-218d-4a99-916d-e1fa5e42efd5 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:54:20.382581 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 647/1291] 10.222.0.18 () {56 vars in 1116 bytes} [Sat Aug 6 21:54:20 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:54:20.388249 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a998281c-f38c-4567-856c-90926e24e563 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:54:20.488428 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a998281c-f38c-4567-856c-90926e24e563 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:54:20.521103 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc24c08-42ed-47cb-9384-e8363155cfa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:54:20.521695 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc24c08-42ed-47cb-9384-e8363155cfa0 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:20.521971 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc24c08-42ed-47cb-9384-e8363155cfa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:20.522242 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc24c08-42ed-47cb-9384-e8363155cfa0 None admin] PATH_INFO: `/v3/services` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:20.522860 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc24c08-42ed-47cb-9384-e8363155cfa0 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:54:20.523159 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc24c08-42ed-47cb-9384-e8363155cfa0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:54:20.524592 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc24c08-42ed-47cb-9384-e8363155cfa0 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:54:20.528885 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 645/1292] 10.222.0.18 () {56 vars in 1116 bytes} [Sat Aug 6 21:54:20 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:54:22.537789 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb23903-0493-4348-a809-e8000924ede4 None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:22.538309 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb23903-0493-4348-a809-e8000924ede4 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:22.538309 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb23903-0493-4348-a809-e8000924ede4 None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:22.539660 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 648/1293] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:54:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:54:22.544980 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d49e49-ec70-4785-8e4c-b9d8e20bad39 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:22.545395 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d49e49-ec70-4785-8e4c-b9d8e20bad39 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:22.545537 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d49e49-ec70-4785-8e4c-b9d8e20bad39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:22.563497 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-67d49e49-ec70-4785-8e4c-b9d8e20bad39 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:54:22.564748 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-67d49e49-ec70-4785-8e4c-b9d8e20bad39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:54:22.594556 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:54:22.630889 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 646/1294] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:54:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 21:54:22.653508 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb23903-0493-4348-a809-e8000924ede4 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:54:22.684367 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:54:22.752846 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb23903-0493-4348-a809-e8000924ede4 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:54:22.786537 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2729f180-696e-4fd8-8668-0670f8bbeb2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:54:22.787141 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2729f180-696e-4fd8-8668-0670f8bbeb2f None admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:22.787420 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2729f180-696e-4fd8-8668-0670f8bbeb2f None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:22.787691 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2729f180-696e-4fd8-8668-0670f8bbeb2f None admin] PATH_INFO: `/v3/services` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:22.788339 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2729f180-696e-4fd8-8668-0670f8bbeb2f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:54:22.789725 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2729f180-696e-4fd8-8668-0670f8bbeb2f None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:54:22.987605 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 649/1295] 10.222.0.18 () {60 vars in 1141 bytes} [Sat Aug 6 21:54:22 2022] POST /identity/v3/services => generated 246 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 21:54:25.013840 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb57b57-24be-47b5-bc37-4aa167aa784b None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:25.014407 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb57b57-24be-47b5-bc37-4aa167aa784b None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:25.014407 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb57b57-24be-47b5-bc37-4aa167aa784b None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:25.015698 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 647/1296] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:54:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:54:25.021317 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f20b4e-ba50-404c-9564-10fada0ba6af None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:25.021698 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f20b4e-ba50-404c-9564-10fada0ba6af None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:25.021952 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f20b4e-ba50-404c-9564-10fada0ba6af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:25.040545 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-a2f20b4e-ba50-404c-9564-10fada0ba6af None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:54:25.041733 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-a2f20b4e-ba50-404c-9564-10fada0ba6af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:54:25.113611 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 650/1297] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:54:25 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 21:54:25.136648 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb57b57-24be-47b5-bc37-4aa167aa784b None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:54:25.237189 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb57b57-24be-47b5-bc37-4aa167aa784b None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:54:25.272308 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554e1339-6d73-4efb-899a-72849ed68b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:54:25.272308 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554e1339-6d73-4efb-899a-72849ed68b04 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:25.272308 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554e1339-6d73-4efb-899a-72849ed68b04 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:25.272308 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554e1339-6d73-4efb-899a-72849ed68b04 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:25.272732 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554e1339-6d73-4efb-899a-72849ed68b04 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:54:25.273444 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554e1339-6d73-4efb-899a-72849ed68b04 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:54:25.281808 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-554e1339-6d73-4efb-899a-72849ed68b04 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 06 21:54:25.281808 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:54:25.281808 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 21:54:25.281808 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 21:54:25.281808 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 21:54:25.281808 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 21:54:25.281808 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 21:54:25.281808 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 21:54:25.281808 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 06 21:54:25.281808 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:54:25.281808 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 21:54:25.281808 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:54:25.281808 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:54:25.281808 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:54:25.281808 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:54:25.281808 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:54:25.281808 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:54:25.283000 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:54:25.283000 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:54:25.283000 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:54:25.283000 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:54:25.283000 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:54:25.283000 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:54:25.283000 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 21:54:25.283000 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 21:54:25.283000 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 21:54:25.283000 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 21:54:25.283000 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:54:25.283000 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:54:25.283000 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:54:25.283000 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:54:25.283000 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:54:25.283000 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:54:25.284015 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:54:25.284015 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:54:25.284015 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:54:25.284015 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:54:25.284015 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:54:25.284015 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:54:25.284015 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:54:25.284015 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:54:25.284015 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:54:25.284015 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:54:25.284015 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 21:54:25.284015 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 21:54:25.284015 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 06 21:54:25.284015 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:54:25.284015 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 648/1298] 10.222.0.18 () {56 vars in 1117 bytes} [Sat Aug 6 21:54:25 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:54:25.289555 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f20b4e-ba50-404c-9564-10fada0ba6af None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:54:25.308419 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:54:25.389310 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f20b4e-ba50-404c-9564-10fada0ba6af None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:54:25.422735 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8b264a-1929-4bf9-a970-a998102d0031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:54:25.423325 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8b264a-1929-4bf9-a970-a998102d0031 None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:25.423600 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8b264a-1929-4bf9-a970-a998102d0031 None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:25.423869 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8b264a-1929-4bf9-a970-a998102d0031 None admin] PATH_INFO: `/v3/services` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:25.424541 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8b264a-1929-4bf9-a970-a998102d0031 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:54:25.424840 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8b264a-1929-4bf9-a970-a998102d0031 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:54:25.426281 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8b264a-1929-4bf9-a970-a998102d0031 None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:54:25.431182 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 651/1299] 10.222.0.18 () {56 vars in 1116 bytes} [Sat Aug 6 21:54:25 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:54:25.437342 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554e1339-6d73-4efb-899a-72849ed68b04 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:54:25.469519 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:54:25.535472 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554e1339-6d73-4efb-899a-72849ed68b04 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:54:25.567382 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cac2b8-9d79-460f-b736-21f024e8cf0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:54:25.567963 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cac2b8-9d79-460f-b736-21f024e8cf0a None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:25.568242 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cac2b8-9d79-460f-b736-21f024e8cf0a None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:25.568540 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cac2b8-9d79-460f-b736-21f024e8cf0a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:25.569231 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cac2b8-9d79-460f-b736-21f024e8cf0a None admin] RBAC: Adding query filter params (interface=public, service_id=47b7da5b640f456c91a23c8a3cabf126, region_id=RegionOne) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:54:25.569536 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cac2b8-9d79-460f-b736-21f024e8cf0a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:54:25.570900 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cac2b8-9d79-460f-b736-21f024e8cf0a None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:54:25.575696 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 649/1300] 10.222.0.18 () {56 vars in 1251 bytes} [Sat Aug 6 21:54:25 2022] GET /identity/v3/endpoints?service_id=47b7da5b640f456c91a23c8a3cabf126&interface=public®ion_id=RegionOne => generated 194 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:54:25.581280 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8b264a-1929-4bf9-a970-a998102d0031 None admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:54:25.661445 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:54:25.678495 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8b264a-1929-4bf9-a970-a998102d0031 None admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:54:25.711969 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660df276-5bb8-4482-abc6-1394a31b667c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:54:25.712585 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660df276-5bb8-4482-abc6-1394a31b667c None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:25.712860 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660df276-5bb8-4482-abc6-1394a31b667c None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:25.713125 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660df276-5bb8-4482-abc6-1394a31b667c None admin] PATH_INFO: `/v3/services` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:25.713718 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660df276-5bb8-4482-abc6-1394a31b667c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:54:25.715054 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660df276-5bb8-4482-abc6-1394a31b667c None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:54:25.721825 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 652/1301] 10.222.0.18 () {56 vars in 1087 bytes} [Sat Aug 6 21:54:25 2022] GET /identity/v3/services => generated 2191 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 21:54:27.704494 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd78113-7688-49a4-bea8-7f0cf0ff049c None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:27.705010 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd78113-7688-49a4-bea8-7f0cf0ff049c None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:27.705010 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd78113-7688-49a4-bea8-7f0cf0ff049c None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:27.706305 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 650/1302] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 21:54:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 21:54:27.711531 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799d4ecd-6f8c-41af-8f84-667cd95a17f4 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:27.711936 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799d4ecd-6f8c-41af-8f84-667cd95a17f4 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:27.712076 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799d4ecd-6f8c-41af-8f84-667cd95a17f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:27.847254 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-799d4ecd-6f8c-41af-8f84-667cd95a17f4 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 21:54:27.848656 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-799d4ecd-6f8c-41af-8f84-667cd95a17f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 21:54:27.915264 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 653/1303] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 21:54:27 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 21:54:27.938635 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd78113-7688-49a4-bea8-7f0cf0ff049c None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:54:28.033527 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:54:28.040648 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd78113-7688-49a4-bea8-7f0cf0ff049c None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:54:28.073820 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74327b2f-c4b8-40c9-9d09-2472763b4133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:54:28.074441 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74327b2f-c4b8-40c9-9d09-2472763b4133 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:28.074716 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74327b2f-c4b8-40c9-9d09-2472763b4133 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:28.074985 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74327b2f-c4b8-40c9-9d09-2472763b4133 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:28.075556 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74327b2f-c4b8-40c9-9d09-2472763b4133 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:54:28.076967 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74327b2f-c4b8-40c9-9d09-2472763b4133 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:54:28.086717 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-74327b2f-c4b8-40c9-9d09-2472763b4133 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 06 21:54:28.086717 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:54:28.086717 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 21:54:28.086717 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 21:54:28.086717 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 21:54:28.086717 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 21:54:28.086717 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 21:54:28.086717 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 21:54:28.086717 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 06 21:54:28.086717 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:54:28.086717 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 21:54:28.086717 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:54:28.086717 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 21:54:28.086717 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 06 21:54:28.086717 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 21:54:28.086717 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 06 21:54:28.086717 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 06 21:54:28.090984 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 21:54:28.090984 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 21:54:28.090984 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 06 21:54:28.090984 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 06 21:54:28.090984 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 21:54:28.090984 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 21:54:28.090984 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 21:54:28.090984 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 21:54:28.090984 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 21:54:28.090984 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 21:54:28.090984 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 21:54:28.090984 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 21:54:28.090984 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 21:54:28.090984 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 21:54:28.090984 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 21:54:28.090984 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 21:54:28.092018 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 21:54:28.092018 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Aug 06 21:54:28.092018 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 21:54:28.092018 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Aug 06 21:54:28.092018 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 21:54:28.092018 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 21:54:28.092018 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 21:54:28.092018 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Aug 06 21:54:28.092018 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 21:54:28.092018 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Aug 06 21:54:28.092018 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 21:54:28.092018 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 21:54:28.092018 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 06 21:54:28.092018 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Aug 06 21:54:28.092018 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 651/1304] 10.222.0.18 () {56 vars in 1117 bytes} [Sat Aug 6 21:54:27 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 21:54:28.093774 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799d4ecd-6f8c-41af-8f84-667cd95a17f4 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:54:28.169219 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:54:28.191102 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799d4ecd-6f8c-41af-8f84-667cd95a17f4 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:54:28.224066 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3e4b1f-cf4c-44fd-b3ae-4e03533dd77d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:54:28.224681 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3e4b1f-cf4c-44fd-b3ae-4e03533dd77d None admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:28.224960 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3e4b1f-cf4c-44fd-b3ae-4e03533dd77d None admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:28.225229 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3e4b1f-cf4c-44fd-b3ae-4e03533dd77d None admin] PATH_INFO: `/v3/services` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:28.225851 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3e4b1f-cf4c-44fd-b3ae-4e03533dd77d None admin] RBAC: Adding query filter params (name=placement) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 21:54:28.226149 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3e4b1f-cf4c-44fd-b3ae-4e03533dd77d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:54:28.227589 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3e4b1f-cf4c-44fd-b3ae-4e03533dd77d None admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:54:28.232851 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 654/1305] 10.222.0.18 () {56 vars in 1116 bytes} [Sat Aug 6 21:54:28 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 21:54:28.238803 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74327b2f-c4b8-40c9-9d09-2472763b4133 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 21:54:28.336697 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74327b2f-c4b8-40c9-9d09-2472763b4133 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 21:54:28.378486 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b57206-ddff-4ec1-a9fe-fc484b197a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 21:54:28.379081 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b57206-ddff-4ec1-a9fe-fc484b197a0d None admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 21:54:28.380059 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b57206-ddff-4ec1-a9fe-fc484b197a0d None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 21:54:28.380478 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b57206-ddff-4ec1-a9fe-fc484b197a0d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 21:54:28.381095 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b57206-ddff-4ec1-a9fe-fc484b197a0d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 21:54:28.382496 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b57206-ddff-4ec1-a9fe-fc484b197a0d None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 21:54:28.492834 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 652/1306] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 21:54:28 2022] POST /identity/v3/endpoints => generated 331 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 21:54:38.353194 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=30 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 21:54:38.487472 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:01:59.314173 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d978c71-a2b2-4cf3-a11e-e8de20016710 None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:01:59.314569 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d978c71-a2b2-4cf3-a11e-e8de20016710 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:01:59.314798 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d978c71-a2b2-4cf3-a11e-e8de20016710 None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:01:59.321990 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a21540-5714-418f-b2e6-d7f6aa505276 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:01:59.322415 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a21540-5714-418f-b2e6-d7f6aa505276 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:01:59.322495 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a21540-5714-418f-b2e6-d7f6aa505276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:01:59.324470 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 655/1307] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 22:01:59 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:01:59.409865 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-b8a21540-5714-418f-b2e6-d7f6aa505276 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:01:59.411165 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-b8a21540-5714-418f-b2e6-d7f6aa505276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:01:59.541064 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 653/1308] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 22:01:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:01:59.581681 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2468d965-1c6e-46ad-bbf6-e71cf9edf260 None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:01:59.581951 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2468d965-1c6e-46ad-bbf6-e71cf9edf260 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:01:59.582221 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2468d965-1c6e-46ad-bbf6-e71cf9edf260 None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:01:59.583893 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 656/1309] 10.222.0.18 () {54 vars in 965 bytes} [Sat Aug 6 22:01:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:01:59.605871 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bbd0de-ea9d-46a9-b917-e9c93bff994c None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:01:59.605871 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bbd0de-ea9d-46a9-b917-e9c93bff994c None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:01:59.605871 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bbd0de-ea9d-46a9-b917-e9c93bff994c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:01:59.689915 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-d0bbd0de-ea9d-46a9-b917-e9c93bff994c None None] MFA Rules not processed for user `5ceaeadc843c4186b25114b28607fe22`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:01:59.693810 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-d0bbd0de-ea9d-46a9-b917-e9c93bff994c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:01:59.721597 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:01:59.795572 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 654/1310] 10.222.0.18 () {58 vars in 1062 bytes} [Sat Aug 6 22:01:59 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:01:59.804298 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2468d965-1c6e-46ad-bbf6-e71cf9edf260 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:01:59.806627 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-2468d965-1c6e-46ad-bbf6-e71cf9edf260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:01:59.947701 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2468d965-1c6e-46ad-bbf6-e71cf9edf260 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:02:00.047488 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b302e5-f02f-4106-838d-fa75f920d3d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:02:00.048494 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b302e5-f02f-4106-838d-fa75f920d3d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:02:00.048975 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b302e5-f02f-4106-838d-fa75f920d3d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:02:00.049430 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b302e5-f02f-4106-838d-fa75f920d3d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:02:00.051510 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-49b302e5-f02f-4106-838d-fa75f920d3d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:02:00.117985 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:02:00.181184 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b302e5-f02f-4106-838d-fa75f920d3d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:02:00.183162 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b302e5-f02f-4106-838d-fa75f920d3d7 service neutron] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:02:00.219630 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 657/1311] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:01:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:02:02.624316 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b21e47f-f203-4390-b5f0-333cf7e561f0 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:02:02.624905 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b21e47f-f203-4390-b5f0-333cf7e561f0 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:02:02.624905 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b21e47f-f203-4390-b5f0-333cf7e561f0 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:02:02.626237 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 655/1312] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 22:02:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:02:02.631344 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0436d589-223b-4730-96b6-a3c541d6d180 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:02:02.631767 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0436d589-223b-4730-96b6-a3c541d6d180 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:02:02.631897 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0436d589-223b-4730-96b6-a3c541d6d180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:02:02.714504 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-0436d589-223b-4730-96b6-a3c541d6d180 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:02:02.715719 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-0436d589-223b-4730-96b6-a3c541d6d180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:02:02.818647 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 658/1313] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 22:02:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:02:02.858525 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b21e47f-f203-4390-b5f0-333cf7e561f0 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:02:02.860219 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-4b21e47f-f203-4390-b5f0-333cf7e561f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:02:02.968574 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b21e47f-f203-4390-b5f0-333cf7e561f0 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:02:03.039954 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:02:03.042929 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72591c9-5347-4b5e-8d9e-479223a18733 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:02:03.043548 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72591c9-5347-4b5e-8d9e-479223a18733 service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:02:03.043826 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72591c9-5347-4b5e-8d9e-479223a18733 service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:02:03.044094 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72591c9-5347-4b5e-8d9e-479223a18733 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:02:03.045809 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-a72591c9-5347-4b5e-8d9e-479223a18733 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:02:03.158860 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72591c9-5347-4b5e-8d9e-479223a18733 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:02:03.160307 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72591c9-5347-4b5e-8d9e-479223a18733 service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:02:03.178293 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 656/1314] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:02:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:02:05.672654 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ca1f44-c574-4b7f-bc82-3e0f7e575235 None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:02:05.673214 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ca1f44-c574-4b7f-bc82-3e0f7e575235 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:02:05.673214 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ca1f44-c574-4b7f-bc82-3e0f7e575235 None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:02:05.674587 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 659/1315] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 22:02:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:02:05.679942 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07a9c0e-8f15-448e-8282-30fd8da234e0 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:02:05.680367 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07a9c0e-8f15-448e-8282-30fd8da234e0 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:02:05.680488 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07a9c0e-8f15-448e-8282-30fd8da234e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:02:05.746488 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-d07a9c0e-8f15-448e-8282-30fd8da234e0 None None] MFA Rules not processed for user `b7b3753a7de94d39b55d2106b904b48d`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:02:05.747699 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-d07a9c0e-8f15-448e-8282-30fd8da234e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:02:05.799353 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:02:05.843677 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 657/1316] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 22:02:05 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:02:05.883792 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ca1f44-c574-4b7f-bc82-3e0f7e575235 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:02:05.885381 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-85ca1f44-c574-4b7f-bc82-3e0f7e575235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:02:05.895046 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:02:05.989150 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ca1f44-c574-4b7f-bc82-3e0f7e575235 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:02:06.067071 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5b62b5-9d9f-4a50-b794-1c333dbfd6ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:02:06.067671 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5b62b5-9d9f-4a50-b794-1c333dbfd6ec service neutron] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:02:06.067946 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5b62b5-9d9f-4a50-b794-1c333dbfd6ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:02:06.068212 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5b62b5-9d9f-4a50-b794-1c333dbfd6ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:02:06.070410 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-ec5b62b5-9d9f-4a50-b794-1c333dbfd6ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:02:06.130820 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:02:06.175996 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5b62b5-9d9f-4a50-b794-1c333dbfd6ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:02:06.177442 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5b62b5-9d9f-4a50-b794-1c333dbfd6ec service neutron] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:02:06.196586 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 660/1317] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:02:05 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:02:09.064119 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0cfed3-02df-4c54-97b5-9216002e8def None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:02:09.064688 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0cfed3-02df-4c54-97b5-9216002e8def None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:02:09.064688 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0cfed3-02df-4c54-97b5-9216002e8def None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:02:09.066008 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 658/1318] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 22:02:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:02:09.071664 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35782b30-5226-4eef-90ab-383c6eb3aaab None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:02:09.072106 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35782b30-5226-4eef-90ab-383c6eb3aaab None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:02:09.072214 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35782b30-5226-4eef-90ab-383c6eb3aaab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:02:09.137452 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-35782b30-5226-4eef-90ab-383c6eb3aaab None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:02:09.138641 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-35782b30-5226-4eef-90ab-383c6eb3aaab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:02:09.231658 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 661/1319] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 22:02:09 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:02:09.271320 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0cfed3-02df-4c54-97b5-9216002e8def None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:02:09.272956 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-2e0cfed3-02df-4c54-97b5-9216002e8def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:02:09.376360 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0cfed3-02df-4c54-97b5-9216002e8def None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:02:09.448694 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bddbda-a33b-4c3e-82c6-8f31b16fc25c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:02:09.449288 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bddbda-a33b-4c3e-82c6-8f31b16fc25c service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:02:09.449562 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bddbda-a33b-4c3e-82c6-8f31b16fc25c service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:02:09.449825 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bddbda-a33b-4c3e-82c6-8f31b16fc25c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:02:09.451397 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-89bddbda-a33b-4c3e-82c6-8f31b16fc25c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:02:09.458593 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:02:09.558865 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bddbda-a33b-4c3e-82c6-8f31b16fc25c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:02:09.560299 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bddbda-a33b-4c3e-82c6-8f31b16fc25c service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:02:09.577726 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 659/1320] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:02:09 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:02:12.802138 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b326e98b-cc02-4726-ac77-4ac46868b0af None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:02:12.802736 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b326e98b-cc02-4726-ac77-4ac46868b0af None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:02:12.802736 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b326e98b-cc02-4726-ac77-4ac46868b0af None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:02:12.805108 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 662/1321] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 22:02:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:02:12.809887 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8cd513-1c1e-45b7-8ce5-a8f8fa47c009 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:02:12.810305 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8cd513-1c1e-45b7-8ce5-a8f8fa47c009 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:02:12.810434 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8cd513-1c1e-45b7-8ce5-a8f8fa47c009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:02:12.877630 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-da8cd513-1c1e-45b7-8ce5-a8f8fa47c009 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:02:12.878820 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-da8cd513-1c1e-45b7-8ce5-a8f8fa47c009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:02:12.936211 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:02:12.978303 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 660/1322] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 22:02:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:02:13.015882 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b326e98b-cc02-4726-ac77-4ac46868b0af None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:02:13.017489 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-b326e98b-cc02-4726-ac77-4ac46868b0af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:02:13.038410 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:02:13.129504 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b326e98b-cc02-4726-ac77-4ac46868b0af None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:02:13.207526 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd32a090-ada6-4859-8f9e-a19e59d477c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:02:13.208124 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd32a090-ada6-4859-8f9e-a19e59d477c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:02:13.208414 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd32a090-ada6-4859-8f9e-a19e59d477c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:02:13.208700 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd32a090-ada6-4859-8f9e-a19e59d477c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:02:13.210667 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-bd32a090-ada6-4859-8f9e-a19e59d477c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:02:13.282506 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:02:13.319460 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd32a090-ada6-4859-8f9e-a19e59d477c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:02:13.320918 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd32a090-ada6-4859-8f9e-a19e59d477c0 service neutron] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:02:13.339896 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 663/1323] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:02:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:02:15.392462 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4554fc-216c-41e5-a07a-e940e149238f None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:02:15.393005 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4554fc-216c-41e5-a07a-e940e149238f None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:02:15.393005 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4554fc-216c-41e5-a07a-e940e149238f None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:02:15.394389 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 661/1324] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 22:02:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:02:15.399546 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c45e7d2-0c27-4ad7-b2f3-4fa27d0a2a64 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:02:15.399959 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c45e7d2-0c27-4ad7-b2f3-4fa27d0a2a64 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:02:15.400130 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c45e7d2-0c27-4ad7-b2f3-4fa27d0a2a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:02:15.468157 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-5c45e7d2-0c27-4ad7-b2f3-4fa27d0a2a64 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:02:15.469394 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-5c45e7d2-0c27-4ad7-b2f3-4fa27d0a2a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:02:15.567183 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 664/1325] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 22:02:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:02:15.604681 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4554fc-216c-41e5-a07a-e940e149238f None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:02:15.606212 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-df4554fc-216c-41e5-a07a-e940e149238f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:02:15.709583 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4554fc-216c-41e5-a07a-e940e149238f None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:02:15.785121 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99234adc-0eb6-4bc8-86bd-f2d63833194f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:02:15.785729 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99234adc-0eb6-4bc8-86bd-f2d63833194f service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:02:15.786004 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99234adc-0eb6-4bc8-86bd-f2d63833194f service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:02:15.786272 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99234adc-0eb6-4bc8-86bd-f2d63833194f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:02:15.787861 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-99234adc-0eb6-4bc8-86bd-f2d63833194f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:02:15.807493 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:02:15.904677 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99234adc-0eb6-4bc8-86bd-f2d63833194f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:02:15.906267 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99234adc-0eb6-4bc8-86bd-f2d63833194f service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:02:15.924824 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 662/1326] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:02:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:02:17.977208 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3668ea3-d6d5-48cf-b440-755340164107 None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:02:17.977806 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3668ea3-d6d5-48cf-b440-755340164107 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:02:17.977806 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3668ea3-d6d5-48cf-b440-755340164107 None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:02:17.979364 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 665/1327] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 22:02:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:02:17.985163 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed2b17b-672a-4154-80a4-d92d778ddf36 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:02:17.985729 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed2b17b-672a-4154-80a4-d92d778ddf36 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:02:17.985729 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed2b17b-672a-4154-80a4-d92d778ddf36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:02:18.054321 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-4ed2b17b-672a-4154-80a4-d92d778ddf36 None None] MFA Rules not processed for user `b7b3753a7de94d39b55d2106b904b48d`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:02:18.055533 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-4ed2b17b-672a-4154-80a4-d92d778ddf36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:02:18.120396 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:02:18.152347 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 663/1328] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 22:02:17 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:02:18.203168 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c224ce3-e309-4895-8148-9819c5c9e8c0 None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:02:18.203445 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c224ce3-e309-4895-8148-9819c5c9e8c0 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:02:18.203718 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c224ce3-e309-4895-8148-9819c5c9e8c0 None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:02:18.205104 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 666/1329] 10.222.0.18 () {54 vars in 965 bytes} [Sat Aug 6 22:02:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:02:18.210893 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0182ea8d-b8fb-4892-836d-742aaa1295e0 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:02:18.211176 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0182ea8d-b8fb-4892-836d-742aaa1295e0 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:02:18.211568 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0182ea8d-b8fb-4892-836d-742aaa1295e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:02:18.292929 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-0182ea8d-b8fb-4892-836d-742aaa1295e0 None None] MFA Rules not processed for user `5ceaeadc843c4186b25114b28607fe22`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:02:18.294169 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-0182ea8d-b8fb-4892-836d-742aaa1295e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:02:18.374154 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 664/1330] 10.222.0.18 () {58 vars in 1062 bytes} [Sat Aug 6 22:02:18 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:02:18.383692 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c224ce3-e309-4895-8148-9819c5c9e8c0 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:02:18.385262 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-4c224ce3-e309-4895-8148-9819c5c9e8c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:02:18.415608 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:02:18.489287 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c224ce3-e309-4895-8148-9819c5c9e8c0 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:02:18.565025 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf3c83c-9673-4a91-8488-bde2d5ba8d55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:02:18.565632 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf3c83c-9673-4a91-8488-bde2d5ba8d55 service neutron] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:02:18.565907 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf3c83c-9673-4a91-8488-bde2d5ba8d55 service neutron] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:02:18.566177 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf3c83c-9673-4a91-8488-bde2d5ba8d55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:02:18.567765 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-8bf3c83c-9673-4a91-8488-bde2d5ba8d55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:02:18.645164 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:02:18.692769 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf3c83c-9673-4a91-8488-bde2d5ba8d55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:02:18.694862 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf3c83c-9673-4a91-8488-bde2d5ba8d55 service neutron] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:02:18.716686 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 667/1331] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:02:18 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:02:21.578110 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa5cebe-3b0f-4e8e-8c88-425471c0424f None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:02:21.578669 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa5cebe-3b0f-4e8e-8c88-425471c0424f None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:02:21.578669 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa5cebe-3b0f-4e8e-8c88-425471c0424f None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:02:21.580001 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 665/1332] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 22:02:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 22:02:21.585909 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37f0a70-51c4-48fe-9814-a9300f47ede5 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:02:21.586349 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37f0a70-51c4-48fe-9814-a9300f47ede5 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:02:21.586461 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37f0a70-51c4-48fe-9814-a9300f47ede5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:02:21.652009 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-e37f0a70-51c4-48fe-9814-a9300f47ede5 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:02:21.653231 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-e37f0a70-51c4-48fe-9814-a9300f47ede5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:02:21.748164 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 668/1333] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 22:02:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:02:21.784454 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa5cebe-3b0f-4e8e-8c88-425471c0424f None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:02:21.786008 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-9fa5cebe-3b0f-4e8e-8c88-425471c0424f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:02:21.859054 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:02:21.896286 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa5cebe-3b0f-4e8e-8c88-425471c0424f None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:02:21.968499 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc38c6ae-6ace-4ced-8050-85f09b67ebf0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:02:21.969092 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc38c6ae-6ace-4ced-8050-85f09b67ebf0 service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:02:21.969366 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc38c6ae-6ace-4ced-8050-85f09b67ebf0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:02:21.969630 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc38c6ae-6ace-4ced-8050-85f09b67ebf0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:02:21.971209 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-fc38c6ae-6ace-4ced-8050-85f09b67ebf0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:02:22.077225 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc38c6ae-6ace-4ced-8050-85f09b67ebf0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:02:22.078651 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc38c6ae-6ace-4ced-8050-85f09b67ebf0 service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:02:22.088529 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:02:22.096263 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 666/1334] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:02:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:02:24.138299 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dc0111-c50e-41d5-b6c3-d2e9b96165df None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:02:24.138833 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dc0111-c50e-41d5-b6c3-d2e9b96165df None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:02:24.138833 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dc0111-c50e-41d5-b6c3-d2e9b96165df None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:02:24.140199 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 669/1335] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 22:02:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:02:24.145227 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e059558-4d36-4fd6-9fcf-a5e3777d8dfe None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:02:24.145650 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e059558-4d36-4fd6-9fcf-a5e3777d8dfe None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:02:24.145778 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e059558-4d36-4fd6-9fcf-a5e3777d8dfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:02:24.211879 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-9e059558-4d36-4fd6-9fcf-a5e3777d8dfe None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:02:24.213125 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-9e059558-4d36-4fd6-9fcf-a5e3777d8dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:02:24.308484 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 667/1336] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 22:02:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:02:24.348922 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dc0111-c50e-41d5-b6c3-d2e9b96165df None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:02:24.351404 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-84dc0111-c50e-41d5-b6c3-d2e9b96165df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:02:24.392004 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:02:24.453712 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dc0111-c50e-41d5-b6c3-d2e9b96165df None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:02:24.525049 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73f6ae4-eb48-4200-bc03-b70608540141 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:02:24.525639 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f6ae4-eb48-4200-bc03-b70608540141 service neutron] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:02:24.525913 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f6ae4-eb48-4200-bc03-b70608540141 service neutron] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:02:24.526182 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f6ae4-eb48-4200-bc03-b70608540141 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:02:24.527771 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-b73f6ae4-eb48-4200-bc03-b70608540141 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:02:24.609613 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:02:24.641778 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73f6ae4-eb48-4200-bc03-b70608540141 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:02:24.643297 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73f6ae4-eb48-4200-bc03-b70608540141 service neutron] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:02:24.666189 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 670/1337] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:02:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:02:27.286752 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3fc301-d27d-4ddd-8e48-17b2bfe23c7a None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:02:27.287285 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3fc301-d27d-4ddd-8e48-17b2bfe23c7a None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:02:27.287457 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3fc301-d27d-4ddd-8e48-17b2bfe23c7a None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:02:27.289019 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 668/1338] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 22:02:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:02:27.294279 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a53b79-27f5-41f6-ac2c-ea0268641b77 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:02:27.294708 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a53b79-27f5-41f6-ac2c-ea0268641b77 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:02:27.294817 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a53b79-27f5-41f6-ac2c-ea0268641b77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:02:27.360634 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-b6a53b79-27f5-41f6-ac2c-ea0268641b77 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:02:27.361832 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-b6a53b79-27f5-41f6-ac2c-ea0268641b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:02:27.457740 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 671/1339] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 22:02:27 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:02:27.495367 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3fc301-d27d-4ddd-8e48-17b2bfe23c7a None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:02:27.496953 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-ae3fc301-d27d-4ddd-8e48-17b2bfe23c7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:02:27.571290 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:02:27.597909 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3fc301-d27d-4ddd-8e48-17b2bfe23c7a None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:02:27.672564 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c279c1a4-bc3f-4d32-9342-a9e3ade66f5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:02:27.673168 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c279c1a4-bc3f-4d32-9342-a9e3ade66f5c service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:02:27.673444 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c279c1a4-bc3f-4d32-9342-a9e3ade66f5c service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:02:27.673717 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c279c1a4-bc3f-4d32-9342-a9e3ade66f5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:02:27.675314 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-c279c1a4-bc3f-4d32-9342-a9e3ade66f5c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:02:27.782558 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c279c1a4-bc3f-4d32-9342-a9e3ade66f5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:02:27.783929 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c279c1a4-bc3f-4d32-9342-a9e3ade66f5c service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:02:27.802006 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 669/1340] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:02:27 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:02:32.034964 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eebd2c-079c-45c3-982b-f27a5643b8f9 None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:02:32.035455 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eebd2c-079c-45c3-982b-f27a5643b8f9 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:02:32.035548 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eebd2c-079c-45c3-982b-f27a5643b8f9 None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:02:32.036891 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 672/1341] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 22:02:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:02:32.042751 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e8e9b2-9797-474b-9656-eb8e9db35ea0 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:02:32.043508 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e8e9b2-9797-474b-9656-eb8e9db35ea0 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:02:32.043787 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e8e9b2-9797-474b-9656-eb8e9db35ea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:02:32.049321 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:02:32.111026 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-90e8e9b2-9797-474b-9656-eb8e9db35ea0 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:02:32.112231 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-90e8e9b2-9797-474b-9656-eb8e9db35ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:02:32.209758 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 670/1342] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 22:02:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:02:32.249587 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eebd2c-079c-45c3-982b-f27a5643b8f9 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:02:32.251241 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-d2eebd2c-079c-45c3-982b-f27a5643b8f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:02:32.299822 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:02:32.358212 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eebd2c-079c-45c3-982b-f27a5643b8f9 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:02:32.432499 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f895d506-5aab-4a37-b428-f75f421b6b87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:02:32.433101 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f895d506-5aab-4a37-b428-f75f421b6b87 service neutron] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:02:32.433377 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f895d506-5aab-4a37-b428-f75f421b6b87 service neutron] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:02:32.433644 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f895d506-5aab-4a37-b428-f75f421b6b87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:02:32.435228 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-f895d506-5aab-4a37-b428-f75f421b6b87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:02:32.529481 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:02:32.545979 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f895d506-5aab-4a37-b428-f75f421b6b87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:02:32.556230 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f895d506-5aab-4a37-b428-f75f421b6b87 service neutron] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:02:32.567002 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 673/1343] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:02:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:02:35.638460 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd4a0f-853c-4e03-87f6-5683d69f2ccc None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:02:35.639017 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd4a0f-853c-4e03-87f6-5683d69f2ccc None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:02:35.639017 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd4a0f-853c-4e03-87f6-5683d69f2ccc None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:02:35.640323 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 671/1344] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 22:02:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:02:35.645400 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03193077-a3a6-4016-b0f7-dd88a5aa8aa7 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:02:35.645807 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03193077-a3a6-4016-b0f7-dd88a5aa8aa7 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:02:35.645956 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03193077-a3a6-4016-b0f7-dd88a5aa8aa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:02:35.711906 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-03193077-a3a6-4016-b0f7-dd88a5aa8aa7 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:02:35.713126 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-03193077-a3a6-4016-b0f7-dd88a5aa8aa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:02:35.811233 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 674/1345] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 22:02:35 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:02:35.848090 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fd4a0f-853c-4e03-87f6-5683d69f2ccc None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:02:35.850437 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-20fd4a0f-853c-4e03-87f6-5683d69f2ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:02:35.938141 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:02:35.954799 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fd4a0f-853c-4e03-87f6-5683d69f2ccc None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:02:36.030410 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac37ec1-309b-4313-912b-596ac735ff58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:02:36.031017 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac37ec1-309b-4313-912b-596ac735ff58 service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:02:36.031297 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac37ec1-309b-4313-912b-596ac735ff58 service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:02:36.031567 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac37ec1-309b-4313-912b-596ac735ff58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:02:36.033216 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-7ac37ec1-309b-4313-912b-596ac735ff58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:02:36.148569 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac37ec1-309b-4313-912b-596ac735ff58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:02:36.150114 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac37ec1-309b-4313-912b-596ac735ff58 service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:02:36.171804 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 672/1346] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:02:35 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:02:40.393009 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa7307d-015e-4782-af0e-be54d1d337d1 None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:02:40.393575 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa7307d-015e-4782-af0e-be54d1d337d1 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:02:40.393575 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa7307d-015e-4782-af0e-be54d1d337d1 None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:02:40.395032 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 675/1347] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 22:02:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:02:40.400582 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44577a08-bf48-4562-b5a6-2d78dc697aae None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:02:40.401092 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44577a08-bf48-4562-b5a6-2d78dc697aae None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:02:40.401092 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44577a08-bf48-4562-b5a6-2d78dc697aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:02:40.418348 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:02:40.476622 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-44577a08-bf48-4562-b5a6-2d78dc697aae None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:02:40.477863 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-44577a08-bf48-4562-b5a6-2d78dc697aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:02:40.584561 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 673/1348] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 22:02:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:02:40.619443 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa7307d-015e-4782-af0e-be54d1d337d1 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:02:40.621028 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-6fa7307d-015e-4782-af0e-be54d1d337d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:02:40.686248 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:02:40.736110 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa7307d-015e-4782-af0e-be54d1d337d1 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:02:40.815302 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c21afb-ba6f-4fe9-a895-75f102e03c6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:02:40.815909 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c21afb-ba6f-4fe9-a895-75f102e03c6b service neutron] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:02:40.816181 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c21afb-ba6f-4fe9-a895-75f102e03c6b service neutron] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:02:40.816537 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c21afb-ba6f-4fe9-a895-75f102e03c6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:02:40.818151 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-d8c21afb-ba6f-4fe9-a895-75f102e03c6b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:02:40.926195 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:02:40.930060 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c21afb-ba6f-4fe9-a895-75f102e03c6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:02:40.931489 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c21afb-ba6f-4fe9-a895-75f102e03c6b service neutron] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:02:40.950022 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 676/1349] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:02:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:02:45.892124 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcaef7c-7991-4b12-b1cd-fbe34f189377 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:02:45.892124 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcaef7c-7991-4b12-b1cd-fbe34f189377 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:02:45.892680 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcaef7c-7991-4b12-b1cd-fbe34f189377 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:02:45.893986 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 674/1350] 10.222.0.18 () {54 vars in 921 bytes} [Sat Aug 6 22:02:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:02:45.898918 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b873d56c-dca7-455b-ac29-1d188b18b959 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:02:45.899360 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b873d56c-dca7-455b-ac29-1d188b18b959 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:02:45.899504 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b873d56c-dca7-455b-ac29-1d188b18b959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:02:46.046210 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-b873d56c-dca7-455b-ac29-1d188b18b959 None None] MFA Rules not processed for user `680d62a67ae2456f8c129e26728c3297`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:02:46.047640 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-b873d56c-dca7-455b-ac29-1d188b18b959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:02:46.174674 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 677/1351] 10.222.0.18 () {58 vars in 1018 bytes} [Sat Aug 6 22:02:45 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:02:50.372138 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d4bde9-bf86-432f-906c-fef26b94b74a None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:02:50.372686 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d4bde9-bf86-432f-906c-fef26b94b74a None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:02:50.372783 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d4bde9-bf86-432f-906c-fef26b94b74a None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:02:50.374224 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 675/1352] 10.222.0.18 () {54 vars in 921 bytes} [Sat Aug 6 22:02:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:02:50.379317 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e575621b-bde1-4068-bbce-85e5c6f69ffb None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:02:50.379784 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e575621b-bde1-4068-bbce-85e5c6f69ffb None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:02:50.379888 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e575621b-bde1-4068-bbce-85e5c6f69ffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:02:50.437164 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:02:50.461290 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-e575621b-bde1-4068-bbce-85e5c6f69ffb None None] MFA Rules not processed for user `680d62a67ae2456f8c129e26728c3297`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:02:50.462496 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-e575621b-bde1-4068-bbce-85e5c6f69ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:02:50.577662 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=59 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:02:50.577878 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 678/1353] 10.222.0.18 () {58 vars in 1018 bytes} [Sat Aug 6 22:02:50 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:02:51.928825 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576ec5c0-7a39-4c30-9ac6-9f2e8e28c159 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:02:51.929361 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576ec5c0-7a39-4c30-9ac6-9f2e8e28c159 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:02:51.929361 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576ec5c0-7a39-4c30-9ac6-9f2e8e28c159 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:02:51.930851 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 676/1354] 10.222.0.18 () {54 vars in 921 bytes} [Sat Aug 6 22:02:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:02:51.935554 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6959179d-ea77-4d32-9e9a-4f30c159e004 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:02:51.936006 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6959179d-ea77-4d32-9e9a-4f30c159e004 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:02:51.936248 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6959179d-ea77-4d32-9e9a-4f30c159e004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:02:52.013090 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-6959179d-ea77-4d32-9e9a-4f30c159e004 None None] MFA Rules not processed for user `680d62a67ae2456f8c129e26728c3297`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:02:52.014283 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-6959179d-ea77-4d32-9e9a-4f30c159e004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:02:52.056317 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:02:52.120989 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 679/1355] 10.222.0.18 () {58 vars in 1018 bytes} [Sat Aug 6 22:02:51 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 22:02:55.504668 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796965c4-9669-400f-8142-99e980799682 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:02:55.505645 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796965c4-9669-400f-8142-99e980799682 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:02:55.506130 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796965c4-9669-400f-8142-99e980799682 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:02:55.511320 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 677/1356] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 22:02:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:02:55.514295 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fe2ab5-e21c-47d2-82ea-1594c93900b7 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:02:55.514737 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fe2ab5-e21c-47d2-82ea-1594c93900b7 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:02:55.514844 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fe2ab5-e21c-47d2-82ea-1594c93900b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:02:55.591330 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-e4fe2ab5-e21c-47d2-82ea-1594c93900b7 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:02:55.592856 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-e4fe2ab5-e21c-47d2-82ea-1594c93900b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:02:55.699243 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 680/1357] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 22:02:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:02:55.761102 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff037120-5d3f-4117-a30e-817c0411aa9d None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:02:55.761864 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff037120-5d3f-4117-a30e-817c0411aa9d None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:02:55.762362 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff037120-5d3f-4117-a30e-817c0411aa9d None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:02:55.764265 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 678/1358] 10.222.0.18 () {54 vars in 962 bytes} [Sat Aug 6 22:02:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:02:55.772933 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931c464f-e15a-4c98-9c47-1ad8565c1abc None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:02:55.773806 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931c464f-e15a-4c98-9c47-1ad8565c1abc None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:02:55.777024 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931c464f-e15a-4c98-9c47-1ad8565c1abc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:02:55.793596 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:02:55.855828 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-931c464f-e15a-4c98-9c47-1ad8565c1abc None None] MFA Rules not processed for user `2a1319d33e9144859c6010843be312a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:02:55.857373 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-931c464f-e15a-4c98-9c47-1ad8565c1abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:02:55.977679 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 681/1359] 10.222.0.18 () {58 vars in 1059 bytes} [Sat Aug 6 22:02:55 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 22:02:55.985704 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff037120-5d3f-4117-a30e-817c0411aa9d None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:02:55.987199 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-ff037120-5d3f-4117-a30e-817c0411aa9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:02:56.141869 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff037120-5d3f-4117-a30e-817c0411aa9d None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:02:56.223014 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:02:56.266815 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d232aff7-17b4-4e0c-8be4-748fb540ce33 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a1319d33e9144859c6010843be312a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:02:56.267519 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d232aff7-17b4-4e0c-8be4-748fb540ce33 service nova] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:02:56.267834 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d232aff7-17b4-4e0c-8be4-748fb540ce33 service nova] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:02:56.268204 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d232aff7-17b4-4e0c-8be4-748fb540ce33 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:02:56.269893 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-d232aff7-17b4-4e0c-8be4-748fb540ce33 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:02:56.381507 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d232aff7-17b4-4e0c-8be4-748fb540ce33 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:02:56.383546 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d232aff7-17b4-4e0c-8be4-748fb540ce33 service nova] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:02:56.405152 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 679/1360] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:02:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 22:02:58.656537 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81c171f-cb18-4845-8cc9-d5c973bb9fcc None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:02:58.656537 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81c171f-cb18-4845-8cc9-d5c973bb9fcc None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:02:58.656537 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81c171f-cb18-4845-8cc9-d5c973bb9fcc None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:02:58.656537 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 682/1361] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 22:02:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:02:58.657246 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9e9f5b-0cda-4d85-9ff1-8f6bf09dc169 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:02:58.657246 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9e9f5b-0cda-4d85-9ff1-8f6bf09dc169 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:02:58.657246 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9e9f5b-0cda-4d85-9ff1-8f6bf09dc169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:02:58.732146 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-7c9e9f5b-0cda-4d85-9ff1-8f6bf09dc169 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:02:58.733568 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-7c9e9f5b-0cda-4d85-9ff1-8f6bf09dc169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:02:58.777439 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:02:58.845100 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 680/1362] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 22:02:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:02:58.898013 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81c171f-cb18-4845-8cc9-d5c973bb9fcc None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:02:58.898013 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-d81c171f-cb18-4845-8cc9-d5c973bb9fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:02:58.976862 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:02:59.080318 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81c171f-cb18-4845-8cc9-d5c973bb9fcc None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:02:59.203561 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba22e43-3dc9-4ac1-a0bc-ade04e2d589b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a1319d33e9144859c6010843be312a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:02:59.204289 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba22e43-3dc9-4ac1-a0bc-ade04e2d589b service nova] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:02:59.204762 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba22e43-3dc9-4ac1-a0bc-ade04e2d589b service nova] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:02:59.205099 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba22e43-3dc9-4ac1-a0bc-ade04e2d589b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:02:59.206759 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-eba22e43-3dc9-4ac1-a0bc-ade04e2d589b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:02:59.231245 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:02:59.301728 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba22e43-3dc9-4ac1-a0bc-ade04e2d589b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:02:59.303471 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba22e43-3dc9-4ac1-a0bc-ade04e2d589b service nova] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:02:59.322458 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 683/1363] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:02:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:03:01.823673 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4632828-4a6d-451f-a1fa-00eb26247db4 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:03:01.824592 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4632828-4a6d-451f-a1fa-00eb26247db4 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:03:01.824871 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4632828-4a6d-451f-a1fa-00eb26247db4 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:03:01.826297 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 681/1364] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 22:03:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:03:01.835522 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635d7968-8b62-40d1-abdf-49ac0000d9ff None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:03:01.835522 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635d7968-8b62-40d1-abdf-49ac0000d9ff None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:03:01.835522 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635d7968-8b62-40d1-abdf-49ac0000d9ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:03:01.909379 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-635d7968-8b62-40d1-abdf-49ac0000d9ff None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:03:01.911084 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-635d7968-8b62-40d1-abdf-49ac0000d9ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:03:02.017684 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:03:02.032698 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 684/1365] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 22:03:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:03:02.067113 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adbcd23-bc5c-4dbb-894b-ae6e84086668 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:03:02.067398 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adbcd23-bc5c-4dbb-894b-ae6e84086668 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:03:02.067670 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adbcd23-bc5c-4dbb-894b-ae6e84086668 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:03:02.069166 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 682/1366] 10.222.0.18 () {54 vars in 962 bytes} [Sat Aug 6 22:03:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:03:02.075789 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc49ba-4ddb-4deb-8b5d-7ef34952fb5c None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:03:02.076070 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc49ba-4ddb-4deb-8b5d-7ef34952fb5c None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:03:02.076351 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc49ba-4ddb-4deb-8b5d-7ef34952fb5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:03:02.158162 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-9abc49ba-4ddb-4deb-8b5d-7ef34952fb5c None None] MFA Rules not processed for user `2a1319d33e9144859c6010843be312a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:03:02.159263 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-9abc49ba-4ddb-4deb-8b5d-7ef34952fb5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:03:02.287371 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 685/1367] 10.222.0.18 () {58 vars in 1059 bytes} [Sat Aug 6 22:03:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:03:02.296126 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adbcd23-bc5c-4dbb-894b-ae6e84086668 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:03:02.298714 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-1adbcd23-bc5c-4dbb-894b-ae6e84086668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:03:02.463362 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adbcd23-bc5c-4dbb-894b-ae6e84086668 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:03:02.485201 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:03:02.582888 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de166fa-b68f-4729-8b7e-7b5ef54fa4b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a1319d33e9144859c6010843be312a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:03:02.583494 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de166fa-b68f-4729-8b7e-7b5ef54fa4b6 service nova] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:03:02.583769 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de166fa-b68f-4729-8b7e-7b5ef54fa4b6 service nova] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:03:02.584035 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de166fa-b68f-4729-8b7e-7b5ef54fa4b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:03:02.585697 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-0de166fa-b68f-4729-8b7e-7b5ef54fa4b6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:03:02.683890 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de166fa-b68f-4729-8b7e-7b5ef54fa4b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:03:02.685567 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de166fa-b68f-4729-8b7e-7b5ef54fa4b6 service nova] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:03:02.705227 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 683/1368] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:03:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:03:03.661250 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36e4f7b-59c7-4e57-baea-07341817a686 None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:03:03.661769 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36e4f7b-59c7-4e57-baea-07341817a686 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:03:03.661769 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36e4f7b-59c7-4e57-baea-07341817a686 None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:03:03.663186 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 686/1369] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 22:03:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:03:03.668243 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b299db-772a-4c86-8b2c-ac11feee60e4 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:03:03.668693 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b299db-772a-4c86-8b2c-ac11feee60e4 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:03:03.668888 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b299db-772a-4c86-8b2c-ac11feee60e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:03:03.702540 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:03:03.741949 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-07b299db-772a-4c86-8b2c-ac11feee60e4 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:03:03.743188 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-07b299db-772a-4c86-8b2c-ac11feee60e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:03:03.860588 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 684/1370] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 22:03:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:03:04.213282 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc3dcb-6f2f-446c-9eab-a63878170372 None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:03:04.213282 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc3dcb-6f2f-446c-9eab-a63878170372 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:03:04.213282 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc3dcb-6f2f-446c-9eab-a63878170372 None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:03:04.215933 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 687/1371] 10.222.0.18 () {54 vars in 964 bytes} [Sat Aug 6 22:03:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:03:04.219063 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f3a6f2-9a49-4445-971d-7c786632f474 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:03:04.219354 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f3a6f2-9a49-4445-971d-7c786632f474 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:03:04.219630 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f3a6f2-9a49-4445-971d-7c786632f474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:03:04.300613 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-04f3a6f2-9a49-4445-971d-7c786632f474 None None] MFA Rules not processed for user `aa71c71de263478da234f3c5716a6e5f`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:03:04.301889 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-04f3a6f2-9a49-4445-971d-7c786632f474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:03:04.355023 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:03:04.385482 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 685/1372] 10.222.0.18 () {58 vars in 1061 bytes} [Sat Aug 6 22:03:04 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:03:04.393387 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cc3dcb-6f2f-446c-9eab-a63878170372 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:03:04.395860 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-24cc3dcb-6f2f-446c-9eab-a63878170372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:03:04.452091 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:03:04.509291 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cc3dcb-6f2f-446c-9eab-a63878170372 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:03:04.588807 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63ec7b3-e667-444b-98bb-05544aabb358 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa71c71de263478da234f3c5716a6e5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:03:04.589420 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63ec7b3-e667-444b-98bb-05544aabb358 service cinder] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:03:04.589700 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63ec7b3-e667-444b-98bb-05544aabb358 service cinder] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:03:04.589969 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63ec7b3-e667-444b-98bb-05544aabb358 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:03:04.591571 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-d63ec7b3-e667-444b-98bb-05544aabb358 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:03:04.695958 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:03:04.710877 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63ec7b3-e667-444b-98bb-05544aabb358 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:03:04.712349 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63ec7b3-e667-444b-98bb-05544aabb358 service cinder] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:03:04.731585 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 688/1373] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:03:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:03:05.044650 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69151b5-7241-4505-a1b4-2f85489973b1 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:03:05.045263 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69151b5-7241-4505-a1b4-2f85489973b1 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:03:05.045263 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69151b5-7241-4505-a1b4-2f85489973b1 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:03:05.053061 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 686/1374] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 22:03:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:03:05.053341 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c228c6-ce85-48a4-a284-5e19eb73ca47 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:03:05.053827 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c228c6-ce85-48a4-a284-5e19eb73ca47 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:03:05.054376 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c228c6-ce85-48a4-a284-5e19eb73ca47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:03:05.144078 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-85c228c6-ce85-48a4-a284-5e19eb73ca47 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:03:05.146977 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-85c228c6-ce85-48a4-a284-5e19eb73ca47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:03:05.247959 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 689/1375] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 22:03:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:03:05.292542 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69151b5-7241-4505-a1b4-2f85489973b1 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:03:05.294056 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-f69151b5-7241-4505-a1b4-2f85489973b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:03:05.447638 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69151b5-7241-4505-a1b4-2f85489973b1 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:03:05.500416 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:03:05.561906 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30de5797-4423-4431-88f8-168cda8c4a89 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a1319d33e9144859c6010843be312a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:03:05.562507 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30de5797-4423-4431-88f8-168cda8c4a89 service nova] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:03:05.562835 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30de5797-4423-4431-88f8-168cda8c4a89 service nova] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:03:05.563108 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30de5797-4423-4431-88f8-168cda8c4a89 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:03:05.564745 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-30de5797-4423-4431-88f8-168cda8c4a89 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:03:05.653721 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30de5797-4423-4431-88f8-168cda8c4a89 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:03:05.655435 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30de5797-4423-4431-88f8-168cda8c4a89 service nova] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:03:05.673749 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 687/1376] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:03:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:03:07.857778 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a9db11-d096-4c84-be5b-57f1e0db4887 None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:03:07.858292 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a9db11-d096-4c84-be5b-57f1e0db4887 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:03:07.858292 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a9db11-d096-4c84-be5b-57f1e0db4887 None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:03:07.859581 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 690/1377] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 22:03:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:03:07.865648 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd37e25-84fe-4014-89c7-fbd9ec0b9058 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:03:07.866067 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd37e25-84fe-4014-89c7-fbd9ec0b9058 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:03:07.866178 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd37e25-84fe-4014-89c7-fbd9ec0b9058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:03:07.942305 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-3fd37e25-84fe-4014-89c7-fbd9ec0b9058 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:03:07.943434 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-3fd37e25-84fe-4014-89c7-fbd9ec0b9058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:03:07.950652 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:03:08.040604 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 688/1378] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 22:03:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:03:08.084488 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a9db11-d096-4c84-be5b-57f1e0db4887 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:03:08.086113 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-20a9db11-d096-4c84-be5b-57f1e0db4887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:03:08.147158 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:03:08.238917 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a9db11-d096-4c84-be5b-57f1e0db4887 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:03:08.392288 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470a528d-d081-48fc-b73f-24586b6dd0b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a1319d33e9144859c6010843be312a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:03:08.393206 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470a528d-d081-48fc-b73f-24586b6dd0b1 service nova] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:03:08.393481 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470a528d-d081-48fc-b73f-24586b6dd0b1 service nova] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:03:08.393750 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470a528d-d081-48fc-b73f-24586b6dd0b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:03:08.395895 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-470a528d-d081-48fc-b73f-24586b6dd0b1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:03:08.422759 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:03:08.509454 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470a528d-d081-48fc-b73f-24586b6dd0b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:03:08.511083 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470a528d-d081-48fc-b73f-24586b6dd0b1 service nova] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:03:08.530732 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 691/1379] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:03:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:03:10.690791 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff92f50-9d25-4dc3-8355-9a7286ba3c14 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:03:10.691934 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff92f50-9d25-4dc3-8355-9a7286ba3c14 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:03:10.691934 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff92f50-9d25-4dc3-8355-9a7286ba3c14 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:03:10.693523 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 689/1380] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 22:03:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:03:10.698600 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857a065b-736c-42eb-a049-77f934f9a4c8 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:03:10.699041 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857a065b-736c-42eb-a049-77f934f9a4c8 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:03:10.699196 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857a065b-736c-42eb-a049-77f934f9a4c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:03:10.785278 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-857a065b-736c-42eb-a049-77f934f9a4c8 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:03:10.786623 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-857a065b-736c-42eb-a049-77f934f9a4c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:03:10.887654 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:03:10.894062 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 692/1381] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 22:03:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:03:10.940248 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff92f50-9d25-4dc3-8355-9a7286ba3c14 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:03:10.942124 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-fff92f50-9d25-4dc3-8355-9a7286ba3c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:03:11.089724 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:03:11.107464 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff92f50-9d25-4dc3-8355-9a7286ba3c14 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:03:11.247493 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862b4ef2-ab84-4656-ac12-3d1737c6b447 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a1319d33e9144859c6010843be312a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:03:11.248120 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b4ef2-ab84-4656-ac12-3d1737c6b447 service nova] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:03:11.248397 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b4ef2-ab84-4656-ac12-3d1737c6b447 service nova] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:03:11.248696 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b4ef2-ab84-4656-ac12-3d1737c6b447 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:03:11.250489 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-862b4ef2-ab84-4656-ac12-3d1737c6b447 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:03:11.362887 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862b4ef2-ab84-4656-ac12-3d1737c6b447 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:03:11.365007 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862b4ef2-ab84-4656-ac12-3d1737c6b447 service nova] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:03:11.376635 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:03:11.388151 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 690/1382] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:03:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:03:13.939150 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34706fdc-ec58-4373-8332-911c76a7480d None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:03:13.939751 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34706fdc-ec58-4373-8332-911c76a7480d None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:03:13.939919 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34706fdc-ec58-4373-8332-911c76a7480d None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:03:13.941756 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 693/1383] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 22:03:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:03:13.947924 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544be45c-5f15-4e51-9a76-27d4d804f7bf None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:03:13.948373 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544be45c-5f15-4e51-9a76-27d4d804f7bf None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:03:13.948619 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544be45c-5f15-4e51-9a76-27d4d804f7bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:03:14.016563 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-544be45c-5f15-4e51-9a76-27d4d804f7bf None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:03:14.017804 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-544be45c-5f15-4e51-9a76-27d4d804f7bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:03:14.116883 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 691/1384] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 22:03:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 22:03:14.161308 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34706fdc-ec58-4373-8332-911c76a7480d None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:03:14.162821 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-34706fdc-ec58-4373-8332-911c76a7480d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:03:14.326321 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34706fdc-ec58-4373-8332-911c76a7480d None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:03:14.395513 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:03:14.446863 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc07306-5ba8-454e-9401-63c1e995cedf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a1319d33e9144859c6010843be312a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:03:14.447605 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc07306-5ba8-454e-9401-63c1e995cedf service nova] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:03:14.447995 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc07306-5ba8-454e-9401-63c1e995cedf service nova] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:03:14.448365 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc07306-5ba8-454e-9401-63c1e995cedf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:03:14.450156 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-ebc07306-5ba8-454e-9401-63c1e995cedf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:03:14.564245 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc07306-5ba8-454e-9401-63c1e995cedf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:03:14.566501 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc07306-5ba8-454e-9401-63c1e995cedf service nova] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:03:14.589944 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 694/1385] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:03:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:03:17.071423 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226e2f96-5b5f-43d6-a152-7cc649565a07 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:03:17.071998 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226e2f96-5b5f-43d6-a152-7cc649565a07 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:03:17.071998 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226e2f96-5b5f-43d6-a152-7cc649565a07 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:03:17.073325 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 692/1386] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 22:03:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 22:03:17.079065 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a13e03-09b8-40d6-9969-4de9ba3bf38d None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:03:17.079501 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a13e03-09b8-40d6-9969-4de9ba3bf38d None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:03:17.079785 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a13e03-09b8-40d6-9969-4de9ba3bf38d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:03:17.157356 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-a4a13e03-09b8-40d6-9969-4de9ba3bf38d None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:03:17.158573 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-a4a13e03-09b8-40d6-9969-4de9ba3bf38d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:03:17.192076 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:03:17.258819 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 695/1387] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 22:03:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:03:17.304023 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226e2f96-5b5f-43d6-a152-7cc649565a07 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:03:17.305660 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-226e2f96-5b5f-43d6-a152-7cc649565a07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:03:17.390623 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:03:17.461759 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226e2f96-5b5f-43d6-a152-7cc649565a07 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:03:17.580193 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1279ef70-1795-4e07-989c-bed12e0767bf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a1319d33e9144859c6010843be312a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:03:17.580984 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1279ef70-1795-4e07-989c-bed12e0767bf service nova] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:03:17.581337 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1279ef70-1795-4e07-989c-bed12e0767bf service nova] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:03:17.582272 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1279ef70-1795-4e07-989c-bed12e0767bf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:03:17.584363 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-1279ef70-1795-4e07-989c-bed12e0767bf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:03:17.637667 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:03:17.689900 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1279ef70-1795-4e07-989c-bed12e0767bf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:03:17.692077 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1279ef70-1795-4e07-989c-bed12e0767bf service nova] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:03:17.711014 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 693/1388] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:03:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:03:19.915593 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dd57e2-841a-4aa8-b0a9-64a3bc6fc5d7 None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:03:19.916150 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dd57e2-841a-4aa8-b0a9-64a3bc6fc5d7 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:03:19.916150 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dd57e2-841a-4aa8-b0a9-64a3bc6fc5d7 None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:03:19.917428 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 696/1389] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 22:03:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:03:19.922540 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7df7c45-7304-44ad-963c-def5cc070d87 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:03:19.922990 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7df7c45-7304-44ad-963c-def5cc070d87 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:03:19.923103 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7df7c45-7304-44ad-963c-def5cc070d87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:03:19.989679 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-f7df7c45-7304-44ad-963c-def5cc070d87 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:03:19.990869 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-f7df7c45-7304-44ad-963c-def5cc070d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:03:20.089031 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b993d88-9d75-4461-b5ab-ec3ab5bf8860 None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:03:20.089282 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 694/1390] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 22:03:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:03:20.089377 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b993d88-9d75-4461-b5ab-ec3ab5bf8860 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:03:20.089620 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b993d88-9d75-4461-b5ab-ec3ab5bf8860 None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:03:20.091012 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 697/1391] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 22:03:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 22:03:20.098100 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a635a5-682b-4786-90d8-2660b5352f33 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:03:20.098383 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a635a5-682b-4786-90d8-2660b5352f33 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:03:20.098655 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a635a5-682b-4786-90d8-2660b5352f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:03:20.114592 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:03:20.134402 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b993d88-9d75-4461-b5ab-ec3ab5bf8860 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:03:20.135900 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-6b993d88-9d75-4461-b5ab-ec3ab5bf8860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:03:20.165370 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-96a635a5-682b-4786-90d8-2660b5352f33 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:03:20.166633 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-96a635a5-682b-4786-90d8-2660b5352f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:03:20.263516 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 695/1392] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 22:03:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:03:20.283366 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b993d88-9d75-4461-b5ab-ec3ab5bf8860 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:03:20.302533 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:03:20.320982 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a2c078-b742-42dc-ae70-a6b8f111b447 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:03:20.321277 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a2c078-b742-42dc-ae70-a6b8f111b447 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:03:20.321556 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a2c078-b742-42dc-ae70-a6b8f111b447 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:03:20.322902 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 696/1393] 10.222.0.18 () {54 vars in 966 bytes} [Sat Aug 6 22:03:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 22:03:20.327690 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1214135a-aecd-42dc-a83d-bd2d550a9c02 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:03:20.327975 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1214135a-aecd-42dc-a83d-bd2d550a9c02 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:03:20.328250 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1214135a-aecd-42dc-a83d-bd2d550a9c02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:03:20.404084 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56faaddd-6cfa-4fe0-b83e-b63cd524c9cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a1319d33e9144859c6010843be312a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:03:20.404717 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56faaddd-6cfa-4fe0-b83e-b63cd524c9cb service nova] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:03:20.404994 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56faaddd-6cfa-4fe0-b83e-b63cd524c9cb service nova] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:03:20.405264 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56faaddd-6cfa-4fe0-b83e-b63cd524c9cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:03:20.406864 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-56faaddd-6cfa-4fe0-b83e-b63cd524c9cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:03:20.407129 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-1214135a-aecd-42dc-a83d-bd2d550a9c02 None None] MFA Rules not processed for user `c8d38e73e105476e8de858cfa939ba20`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:03:20.408310 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-1214135a-aecd-42dc-a83d-bd2d550a9c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:03:20.443754 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:03:20.484927 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 697/1394] 10.222.0.18 () {58 vars in 1063 bytes} [Sat Aug 6 22:03:20 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 22:03:20.493355 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1214135a-aecd-42dc-a83d-bd2d550a9c02 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:03:20.499499 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56faaddd-6cfa-4fe0-b83e-b63cd524c9cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:03:20.500889 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1214135a-aecd-42dc-a83d-bd2d550a9c02 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:03:20.501134 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56faaddd-6cfa-4fe0-b83e-b63cd524c9cb service nova] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:03:20.520072 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 698/1395] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:03:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:03:20.574319 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7f741f-d092-4149-af5f-b9255159a0d9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8d38e73e105476e8de858cfa939ba20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:03:20.574918 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7f741f-d092-4149-af5f-b9255159a0d9 service glance] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:03:20.575193 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7f741f-d092-4149-af5f-b9255159a0d9 service glance] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:03:20.575462 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7f741f-d092-4149-af5f-b9255159a0d9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:03:20.577117 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-2d7f741f-d092-4149-af5f-b9255159a0d9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:03:20.684364 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:03:20.687907 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7f741f-d092-4149-af5f-b9255159a0d9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:03:20.689439 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7f741f-d092-4149-af5f-b9255159a0d9 service glance] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:03:20.708649 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 698/1396] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:03:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 22:03:21.605148 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dea7392-d785-46eb-91d6-4d6066c44726 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:03:21.605627 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dea7392-d785-46eb-91d6-4d6066c44726 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:03:21.605717 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dea7392-d785-46eb-91d6-4d6066c44726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:03:21.632460 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:03:21.674293 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-7dea7392-d785-46eb-91d6-4d6066c44726 None None] MFA Rules not processed for user `d1b26ba592724dd389584bb800d877a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:03:21.675577 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-7dea7392-d785-46eb-91d6-4d6066c44726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:03:21.765148 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 699/1397] 10.222.0.18 () {58 vars in 1009 bytes} [Sat Aug 6 22:03:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:03:21.814433 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7ae939-087e-40a7-a075-30c774db27a4 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:03:21.814998 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7ae939-087e-40a7-a075-30c774db27a4 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:03:21.814998 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7ae939-087e-40a7-a075-30c774db27a4 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:03:21.817184 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 699/1398] 10.222.0.18 () {54 vars in 943 bytes} [Sat Aug 6 22:03:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:03:21.824602 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256eb889-ac0a-4df4-980a-98703420cce3 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:03:21.825022 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256eb889-ac0a-4df4-980a-98703420cce3 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:03:21.825401 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256eb889-ac0a-4df4-980a-98703420cce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:03:21.905630 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-256eb889-ac0a-4df4-980a-98703420cce3 None None] MFA Rules not processed for user `e7fc9da7505744608dfd3bbd46a0a9d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:03:21.906744 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-256eb889-ac0a-4df4-980a-98703420cce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:03:21.960850 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:03:22.015293 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 700/1399] 10.222.0.18 () {58 vars in 1040 bytes} [Sat Aug 6 22:03:21 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:03:22.024711 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7ae939-087e-40a7-a075-30c774db27a4 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:03:22.026219 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-2a7ae939-087e-40a7-a075-30c774db27a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:03:22.164367 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7ae939-087e-40a7-a075-30c774db27a4 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:03:22.238511 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:03:22.271879 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a90376f-5a20-453d-ab7d-69bec1133f19 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7fc9da7505744608dfd3bbd46a0a9d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:03:22.272517 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a90376f-5a20-453d-ab7d-69bec1133f19 service swift] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:03:22.272794 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a90376f-5a20-453d-ab7d-69bec1133f19 service swift] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:03:22.273058 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a90376f-5a20-453d-ab7d-69bec1133f19 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:03:22.274735 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-3a90376f-5a20-453d-ab7d-69bec1133f19 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:03:22.366530 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a90376f-5a20-453d-ab7d-69bec1133f19 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:03:22.368159 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a90376f-5a20-453d-ab7d-69bec1133f19 service swift] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:03:22.378236 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 700/1400] 10.222.0.18 () {60 vars in 1384 bytes} [Sat Aug 6 22:03:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 22:03:22.764565 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae23783-f23f-4edb-95ac-2186c318b3d0 None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:03:22.764565 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae23783-f23f-4edb-95ac-2186c318b3d0 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:03:22.764565 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae23783-f23f-4edb-95ac-2186c318b3d0 None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:03:22.765539 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 701/1401] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 22:03:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:03:22.774823 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fffd98-6fa7-4029-94d6-54f8128d9c8d None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:03:22.775129 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fffd98-6fa7-4029-94d6-54f8128d9c8d None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:03:22.775538 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fffd98-6fa7-4029-94d6-54f8128d9c8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:03:22.850882 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-58fffd98-6fa7-4029-94d6-54f8128d9c8d None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:03:22.852096 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-58fffd98-6fa7-4029-94d6-54f8128d9c8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:03:22.916377 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:03:22.960336 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 701/1402] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 22:03:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:03:23.007874 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae23783-f23f-4edb-95ac-2186c318b3d0 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:03:23.009778 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-2ae23783-f23f-4edb-95ac-2186c318b3d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:03:23.195638 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae23783-f23f-4edb-95ac-2186c318b3d0 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:03:23.224733 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:03:23.325328 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab79c2c0-ca03-4808-a550-5078f19ef527 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a1319d33e9144859c6010843be312a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:03:23.326057 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab79c2c0-ca03-4808-a550-5078f19ef527 service nova] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:03:23.326403 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab79c2c0-ca03-4808-a550-5078f19ef527 service nova] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:03:23.326739 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab79c2c0-ca03-4808-a550-5078f19ef527 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:03:23.328558 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-ab79c2c0-ca03-4808-a550-5078f19ef527 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:03:23.432955 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab79c2c0-ca03-4808-a550-5078f19ef527 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:03:23.434816 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab79c2c0-ca03-4808-a550-5078f19ef527 service nova] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:03:23.455754 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 702/1403] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:03:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:03:25.690751 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dae2de-19a6-40c5-aad1-53a3f8a59c62 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:03:25.691376 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dae2de-19a6-40c5-aad1-53a3f8a59c62 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:03:25.691463 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dae2de-19a6-40c5-aad1-53a3f8a59c62 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:03:25.692810 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 702/1404] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 22:03:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:03:25.698233 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e2515b-4c26-424a-8f73-385c4ab63b3b None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:03:25.698757 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e2515b-4c26-424a-8f73-385c4ab63b3b None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:03:25.698757 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e2515b-4c26-424a-8f73-385c4ab63b3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:03:25.743937 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:03:25.778723 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-02e2515b-4c26-424a-8f73-385c4ab63b3b None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:03:25.781347 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-02e2515b-4c26-424a-8f73-385c4ab63b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:03:25.896269 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 703/1405] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 22:03:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:03:25.942504 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dae2de-19a6-40c5-aad1-53a3f8a59c62 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:03:25.943975 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-f2dae2de-19a6-40c5-aad1-53a3f8a59c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:03:26.116887 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dae2de-19a6-40c5-aad1-53a3f8a59c62 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:03:26.163534 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:03:26.254202 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fc40a2-9182-4d22-b482-68006c22ef8c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a1319d33e9144859c6010843be312a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:03:26.254811 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fc40a2-9182-4d22-b482-68006c22ef8c service nova] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:03:26.256561 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fc40a2-9182-4d22-b482-68006c22ef8c service nova] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:03:26.256842 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fc40a2-9182-4d22-b482-68006c22ef8c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:03:26.258581 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-73fc40a2-9182-4d22-b482-68006c22ef8c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:03:26.369542 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fc40a2-9182-4d22-b482-68006c22ef8c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:03:26.371185 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fc40a2-9182-4d22-b482-68006c22ef8c service nova] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:03:26.399023 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 703/1406] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:03:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:03:28.598909 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a32db07-e137-45d7-ae55-54540c1cbbbe None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:03:28.599509 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a32db07-e137-45d7-ae55-54540c1cbbbe None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:03:28.599778 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a32db07-e137-45d7-ae55-54540c1cbbbe None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:03:28.601421 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 704/1407] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 22:03:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:03:28.606800 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf6f5fe-a9c5-4161-bf11-52c78a3c3c3c None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:03:28.607208 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf6f5fe-a9c5-4161-bf11-52c78a3c3c3c None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:03:28.607322 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf6f5fe-a9c5-4161-bf11-52c78a3c3c3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:03:28.672654 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:03:28.680664 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-2bf6f5fe-a9c5-4161-bf11-52c78a3c3c3c None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:03:28.682241 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-2bf6f5fe-a9c5-4161-bf11-52c78a3c3c3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:03:28.795334 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 704/1408] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 22:03:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:03:28.841623 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a32db07-e137-45d7-ae55-54540c1cbbbe None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:03:28.843157 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-4a32db07-e137-45d7-ae55-54540c1cbbbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:03:28.957884 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:03:29.004177 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a32db07-e137-45d7-ae55-54540c1cbbbe None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:03:29.128273 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822efd58-7fd8-4354-9a95-0e3b2466bea6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a1319d33e9144859c6010843be312a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:03:29.128929 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822efd58-7fd8-4354-9a95-0e3b2466bea6 service nova] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:03:29.129207 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822efd58-7fd8-4354-9a95-0e3b2466bea6 service nova] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:03:29.129483 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822efd58-7fd8-4354-9a95-0e3b2466bea6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:03:29.131421 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-822efd58-7fd8-4354-9a95-0e3b2466bea6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:03:29.211657 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:03:29.235589 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822efd58-7fd8-4354-9a95-0e3b2466bea6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:03:29.237833 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822efd58-7fd8-4354-9a95-0e3b2466bea6 service nova] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:03:29.258239 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 705/1409] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:03:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:03:38.793034 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=44 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:03:39.255731 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=14 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:03:46.071271 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7d933a-c610-40e6-b5d9-bf0c23dfefbd None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:03:46.071843 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7d933a-c610-40e6-b5d9-bf0c23dfefbd None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:03:46.071843 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7d933a-c610-40e6-b5d9-bf0c23dfefbd None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:03:46.073172 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 705/1410] 10.222.0.18 () {54 vars in 918 bytes} [Sat Aug 6 22:03:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:03:46.078384 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d108e3c-2fc2-4a5d-94ca-88e4f6a17930 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:03:46.078821 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d108e3c-2fc2-4a5d-94ca-88e4f6a17930 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:03:46.078929 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d108e3c-2fc2-4a5d-94ca-88e4f6a17930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:03:46.156488 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-3d108e3c-2fc2-4a5d-94ca-88e4f6a17930 None None] MFA Rules not processed for user `680d62a67ae2456f8c129e26728c3297`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:03:46.157674 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-3d108e3c-2fc2-4a5d-94ca-88e4f6a17930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:03:46.265434 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 706/1411] 10.222.0.18 () {58 vars in 1015 bytes} [Sat Aug 6 22:03:46 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:03:46.279918 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51429464-4195-4d78-8141-eba43a70dda5 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:03:46.280210 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51429464-4195-4d78-8141-eba43a70dda5 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:03:46.280531 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51429464-4195-4d78-8141-eba43a70dda5 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:03:46.281856 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 706/1412] 10.222.0.18 () {54 vars in 918 bytes} [Sat Aug 6 22:03:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:03:46.287201 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b359bac7-23d7-40e6-bd4f-2afa23c0563c None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:03:46.287488 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b359bac7-23d7-40e6-bd4f-2afa23c0563c None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:03:46.287764 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b359bac7-23d7-40e6-bd4f-2afa23c0563c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:03:46.360478 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:03:46.364131 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-b359bac7-23d7-40e6-bd4f-2afa23c0563c None None] MFA Rules not processed for user `2a1319d33e9144859c6010843be312a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:03:46.365360 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-b359bac7-23d7-40e6-bd4f-2afa23c0563c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:03:46.479604 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 707/1413] 10.222.0.18 () {58 vars in 1015 bytes} [Sat Aug 6 22:03:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:03:48.989339 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc32830c-9cb2-4893-ab06-e95130c9d24a None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:03:48.989892 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc32830c-9cb2-4893-ab06-e95130c9d24a None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:03:48.989892 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc32830c-9cb2-4893-ab06-e95130c9d24a None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:03:48.991243 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 707/1414] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 6 22:03:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:03:48.996571 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34ab0cd-ffbc-4152-9f8f-feca9b89eb2a None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:03:48.996956 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34ab0cd-ffbc-4152-9f8f-feca9b89eb2a None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:03:48.997105 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34ab0cd-ffbc-4152-9f8f-feca9b89eb2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:03:49.062836 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-a34ab0cd-ffbc-4152-9f8f-feca9b89eb2a None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:03:49.064059 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-a34ab0cd-ffbc-4152-9f8f-feca9b89eb2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:03:49.120242 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:03:49.162536 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 708/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 6 22:03:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:03:49.205434 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc32830c-9cb2-4893-ab06-e95130c9d24a None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:03:49.206949 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-cc32830c-9cb2-4893-ab06-e95130c9d24a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:03:49.352238 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc32830c-9cb2-4893-ab06-e95130c9d24a None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:03:49.424684 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:03:49.465500 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532ba5f6-86ae-4b8b-914f-1f50f70d4711 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a1319d33e9144859c6010843be312a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:03:49.466086 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532ba5f6-86ae-4b8b-914f-1f50f70d4711 service nova] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:03:49.466362 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532ba5f6-86ae-4b8b-914f-1f50f70d4711 service nova] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:03:49.466630 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532ba5f6-86ae-4b8b-914f-1f50f70d4711 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:03:49.468223 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-532ba5f6-86ae-4b8b-914f-1f50f70d4711 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:03:49.557218 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532ba5f6-86ae-4b8b-914f-1f50f70d4711 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:03:49.558865 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532ba5f6-86ae-4b8b-914f-1f50f70d4711 service nova] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:03:49.576702 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 708/1416] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:03:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:03:50.479458 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8a4f05-3113-4f0c-a726-a423b6bdbe2c None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:03:50.480012 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8a4f05-3113-4f0c-a726-a423b6bdbe2c None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:03:50.480012 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8a4f05-3113-4f0c-a726-a423b6bdbe2c None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:03:50.481361 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 709/1417] 127.0.0.1 () {54 vars in 901 bytes} [Sat Aug 6 22:03:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:03:50.486282 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a2a14d-36a7-439d-aebc-8a4cbade6d27 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:03:50.486686 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a2a14d-36a7-439d-aebc-8a4cbade6d27 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:03:50.486912 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a2a14d-36a7-439d-aebc-8a4cbade6d27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:03:50.562103 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-88a2a14d-36a7-439d-aebc-8a4cbade6d27 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:03:50.563305 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-88a2a14d-36a7-439d-aebc-8a4cbade6d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:03:50.590379 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:03:50.660069 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 709/1418] 127.0.0.1 () {58 vars in 998 bytes} [Sat Aug 6 22:03:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:03:50.809295 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8a4f05-3113-4f0c-a726-a423b6bdbe2c None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:03:50.810824 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-5c8a4f05-3113-4f0c-a726-a423b6bdbe2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:03:50.954414 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8a4f05-3113-4f0c-a726-a423b6bdbe2c None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:03:50.995869 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:03:51.067974 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d90d7f-dedf-4dca-8f5e-79be6736a9dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a1319d33e9144859c6010843be312a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:03:51.068600 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d90d7f-dedf-4dca-8f5e-79be6736a9dc service nova] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:03:51.068877 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d90d7f-dedf-4dca-8f5e-79be6736a9dc service nova] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:03:51.069146 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d90d7f-dedf-4dca-8f5e-79be6736a9dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:03:51.070774 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-72d90d7f-dedf-4dca-8f5e-79be6736a9dc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:03:51.160446 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d90d7f-dedf-4dca-8f5e-79be6736a9dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:03:51.162037 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d90d7f-dedf-4dca-8f5e-79be6736a9dc service nova] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:03:51.180247 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 710/1419] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:03:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:03:52.055190 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbab2a5-c667-4cc7-a813-a58c1174c968 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:03:52.055767 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbab2a5-c667-4cc7-a813-a58c1174c968 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:03:52.055767 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbab2a5-c667-4cc7-a813-a58c1174c968 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:03:52.057108 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 710/1420] 127.0.0.1 () {54 vars in 901 bytes} [Sat Aug 6 22:03:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:03:52.062064 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cf4889-0fb3-4330-accd-6dcfbac7c6b8 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:03:52.062488 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cf4889-0fb3-4330-accd-6dcfbac7c6b8 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:03:52.062582 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cf4889-0fb3-4330-accd-6dcfbac7c6b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:03:52.117081 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:03:52.138810 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-c9cf4889-0fb3-4330-accd-6dcfbac7c6b8 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:03:52.139994 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-c9cf4889-0fb3-4330-accd-6dcfbac7c6b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:03:52.235755 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 711/1421] 127.0.0.1 () {58 vars in 998 bytes} [Sat Aug 6 22:03:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:03:52.397774 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbab2a5-c667-4cc7-a813-a58c1174c968 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:03:52.399376 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-4dbab2a5-c667-4cc7-a813-a58c1174c968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:03:52.543991 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbab2a5-c667-4cc7-a813-a58c1174c968 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:03:52.556260 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:03:52.657185 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcca4c6e-9d3a-428e-b921-0b21cb7b2872 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a1319d33e9144859c6010843be312a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:03:52.657774 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcca4c6e-9d3a-428e-b921-0b21cb7b2872 service nova] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:03:52.658047 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcca4c6e-9d3a-428e-b921-0b21cb7b2872 service nova] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:03:52.658316 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcca4c6e-9d3a-428e-b921-0b21cb7b2872 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:03:52.659887 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-dcca4c6e-9d3a-428e-b921-0b21cb7b2872 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:03:52.747271 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcca4c6e-9d3a-428e-b921-0b21cb7b2872 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:03:52.748946 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcca4c6e-9d3a-428e-b921-0b21cb7b2872 service nova] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:03:52.767274 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 711/1422] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:03:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:03:53.632568 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e6356-97d9-4476-baf2-15f2eebe621d None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:03:53.633079 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e6356-97d9-4476-baf2-15f2eebe621d None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:03:53.633079 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e6356-97d9-4476-baf2-15f2eebe621d None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:03:53.634443 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 712/1423] 127.0.0.1 () {54 vars in 901 bytes} [Sat Aug 6 22:03:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:03:53.639355 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c960be-016c-4087-94ae-6c97cb81a3fc None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:03:53.639776 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c960be-016c-4087-94ae-6c97cb81a3fc None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:03:53.639899 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c960be-016c-4087-94ae-6c97cb81a3fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:03:53.660518 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:03:53.728917 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-52c960be-016c-4087-94ae-6c97cb81a3fc None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:03:53.730172 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-52c960be-016c-4087-94ae-6c97cb81a3fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:03:53.826287 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 712/1424] 127.0.0.1 () {58 vars in 998 bytes} [Sat Aug 6 22:03:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:03:53.977465 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0e6356-97d9-4476-baf2-15f2eebe621d None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:03:53.979256 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-8b0e6356-97d9-4476-baf2-15f2eebe621d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:03:54.089038 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:03:54.124815 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0e6356-97d9-4476-baf2-15f2eebe621d None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:03:54.236824 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b707642c-2763-46eb-9a03-b68dae0d7044 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a1319d33e9144859c6010843be312a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:03:54.237340 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b707642c-2763-46eb-9a03-b68dae0d7044 service nova] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:03:54.237611 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b707642c-2763-46eb-9a03-b68dae0d7044 service nova] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:03:54.237877 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b707642c-2763-46eb-9a03-b68dae0d7044 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:03:54.239455 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-b707642c-2763-46eb-9a03-b68dae0d7044 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:03:54.313256 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:03:54.326469 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b707642c-2763-46eb-9a03-b68dae0d7044 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:03:54.328032 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b707642c-2763-46eb-9a03-b68dae0d7044 service nova] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:03:54.346831 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 713/1425] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:03:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:03:55.216191 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bdabde-6044-478c-ac0e-b5dfb710755d None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:03:55.216765 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bdabde-6044-478c-ac0e-b5dfb710755d None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:03:55.216765 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bdabde-6044-478c-ac0e-b5dfb710755d None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:03:55.218092 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 713/1426] 127.0.0.1 () {54 vars in 901 bytes} [Sat Aug 6 22:03:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:03:55.223748 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d40524-b6da-4ad8-a853-d09a46e71dbc None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:03:55.224172 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d40524-b6da-4ad8-a853-d09a46e71dbc None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:03:55.224441 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d40524-b6da-4ad8-a853-d09a46e71dbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:03:55.299224 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-87d40524-b6da-4ad8-a853-d09a46e71dbc None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:03:55.300424 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-87d40524-b6da-4ad8-a853-d09a46e71dbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:03:55.397017 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 714/1427] 127.0.0.1 () {58 vars in 998 bytes} [Sat Aug 6 22:03:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:03:55.549645 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bdabde-6044-478c-ac0e-b5dfb710755d None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:03:55.551079 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-f9bdabde-6044-478c-ac0e-b5dfb710755d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:03:55.633607 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:03:55.693660 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bdabde-6044-478c-ac0e-b5dfb710755d None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:03:55.806756 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbd6f9f-f7e8-4b31-b557-bcb883570501 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a1319d33e9144859c6010843be312a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:03:55.807282 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbd6f9f-f7e8-4b31-b557-bcb883570501 service nova] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:03:55.807559 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbd6f9f-f7e8-4b31-b557-bcb883570501 service nova] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:03:55.807829 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbd6f9f-f7e8-4b31-b557-bcb883570501 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:03:55.809444 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-3bbd6f9f-f7e8-4b31-b557-bcb883570501 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:03:55.857287 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:03:55.897788 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbd6f9f-f7e8-4b31-b557-bcb883570501 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:03:55.899391 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbd6f9f-f7e8-4b31-b557-bcb883570501 service nova] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:03:55.917636 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 714/1428] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:03:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:03:56.771969 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b051a-1706-4f0f-a994-0446c62f0759 None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:03:56.772569 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b051a-1706-4f0f-a994-0446c62f0759 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:03:56.772569 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b051a-1706-4f0f-a994-0446c62f0759 None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:03:56.773876 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 715/1429] 127.0.0.1 () {54 vars in 901 bytes} [Sat Aug 6 22:03:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:03:56.778742 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3f1032-781d-494f-a991-cec197e1b21a None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:03:56.779141 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3f1032-781d-494f-a991-cec197e1b21a None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:03:56.779291 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3f1032-781d-494f-a991-cec197e1b21a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:03:56.855224 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-6c3f1032-781d-494f-a991-cec197e1b21a None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:03:56.856458 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-6c3f1032-781d-494f-a991-cec197e1b21a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:03:56.953057 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 715/1430] 127.0.0.1 () {58 vars in 998 bytes} [Sat Aug 6 22:03:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 22:03:57.104693 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9b051a-1706-4f0f-a994-0446c62f0759 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:03:57.106107 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-1f9b051a-1706-4f0f-a994-0446c62f0759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:03:57.153534 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:03:57.248926 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9b051a-1706-4f0f-a994-0446c62f0759 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:03:57.360065 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5116b999-6032-4670-a362-986b6250480c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a1319d33e9144859c6010843be312a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:03:57.361161 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5116b999-6032-4670-a362-986b6250480c service nova] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:03:57.361521 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5116b999-6032-4670-a362-986b6250480c service nova] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:03:57.361778 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5116b999-6032-4670-a362-986b6250480c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:03:57.363364 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-5116b999-6032-4670-a362-986b6250480c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:03:57.370355 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:03:57.452325 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5116b999-6032-4670-a362-986b6250480c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:03:57.453943 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5116b999-6032-4670-a362-986b6250480c service nova] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:03:57.472884 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 716/1431] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:03:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:03:58.380025 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7509e9-07ee-4f7d-aec8-a84d210c9e4e None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:03:58.380025 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7509e9-07ee-4f7d-aec8-a84d210c9e4e None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:03:58.380025 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7509e9-07ee-4f7d-aec8-a84d210c9e4e None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:03:58.380025 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 716/1432] 127.0.0.1 () {54 vars in 901 bytes} [Sat Aug 6 22:03:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:03:58.380670 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45d8451-14b4-4af5-9a16-a170e1217c78 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:03:58.380670 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45d8451-14b4-4af5-9a16-a170e1217c78 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:03:58.380670 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45d8451-14b4-4af5-9a16-a170e1217c78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:03:58.446745 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-c45d8451-14b4-4af5-9a16-a170e1217c78 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:03:58.447946 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-c45d8451-14b4-4af5-9a16-a170e1217c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:03:58.514729 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:03:58.542538 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 717/1433] 127.0.0.1 () {58 vars in 998 bytes} [Sat Aug 6 22:03:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:03:58.693774 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7509e9-07ee-4f7d-aec8-a84d210c9e4e None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:03:58.695242 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-9d7509e9-07ee-4f7d-aec8-a84d210c9e4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:03:58.712702 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:03:58.842601 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7509e9-07ee-4f7d-aec8-a84d210c9e4e None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:03:58.928316 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:03:58.952730 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ffbadd-241c-4173-a463-e70676d20cca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a1319d33e9144859c6010843be312a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:03:58.953261 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ffbadd-241c-4173-a463-e70676d20cca service nova] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:03:58.953541 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ffbadd-241c-4173-a463-e70676d20cca service nova] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:03:58.953869 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ffbadd-241c-4173-a463-e70676d20cca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:03:58.955407 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-e3ffbadd-241c-4173-a463-e70676d20cca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:03:59.043500 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ffbadd-241c-4173-a463-e70676d20cca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:03:59.045133 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ffbadd-241c-4173-a463-e70676d20cca service nova] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:03:59.063079 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 717/1434] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:03:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:03:59.935990 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553efdf6-ff9a-4d0c-a188-587a7698c8f2 None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:03:59.936565 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553efdf6-ff9a-4d0c-a188-587a7698c8f2 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:03:59.936565 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553efdf6-ff9a-4d0c-a188-587a7698c8f2 None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:03:59.938215 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 718/1435] 127.0.0.1 () {54 vars in 901 bytes} [Sat Aug 6 22:03:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:03:59.943083 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309fd633-2422-4bc1-b14a-4801ba3983c9 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:03:59.943535 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309fd633-2422-4bc1-b14a-4801ba3983c9 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:03:59.943657 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309fd633-2422-4bc1-b14a-4801ba3983c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:04:00.018612 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-309fd633-2422-4bc1-b14a-4801ba3983c9 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:04:00.019792 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-309fd633-2422-4bc1-b14a-4801ba3983c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:04:00.059996 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:04:00.115504 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 718/1436] 127.0.0.1 () {58 vars in 998 bytes} [Sat Aug 6 22:03:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:04:00.267306 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553efdf6-ff9a-4d0c-a188-587a7698c8f2 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:04:00.268844 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-553efdf6-ff9a-4d0c-a188-587a7698c8f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:04:00.415095 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553efdf6-ff9a-4d0c-a188-587a7698c8f2 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:04:00.468173 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:04:00.527243 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85cf023-b32c-4dac-9301-9b2d42147ec0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a1319d33e9144859c6010843be312a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:04:00.527781 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85cf023-b32c-4dac-9301-9b2d42147ec0 service nova] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:04:00.528060 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85cf023-b32c-4dac-9301-9b2d42147ec0 service nova] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:04:00.528333 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85cf023-b32c-4dac-9301-9b2d42147ec0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:04:00.529991 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-b85cf023-b32c-4dac-9301-9b2d42147ec0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:04:00.618395 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85cf023-b32c-4dac-9301-9b2d42147ec0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:04:00.619964 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85cf023-b32c-4dac-9301-9b2d42147ec0 service nova] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:04:00.638312 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 719/1437] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:04:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:04:01.490420 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17b5f6a-7856-4bd4-b35c-2fd003a9d6b9 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:04:01.490943 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17b5f6a-7856-4bd4-b35c-2fd003a9d6b9 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:04:01.490943 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17b5f6a-7856-4bd4-b35c-2fd003a9d6b9 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:04:01.492223 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 719/1438] 127.0.0.1 () {54 vars in 901 bytes} [Sat Aug 6 22:04:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:04:01.497086 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b387f0-b6f7-4d17-8be5-1d918f289ac6 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:04:01.497479 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b387f0-b6f7-4d17-8be5-1d918f289ac6 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:04:01.497660 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b387f0-b6f7-4d17-8be5-1d918f289ac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:04:01.575622 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-a0b387f0-b6f7-4d17-8be5-1d918f289ac6 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:04:01.576894 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-a0b387f0-b6f7-4d17-8be5-1d918f289ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:04:01.578890 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:04:01.674437 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 720/1439] 127.0.0.1 () {58 vars in 998 bytes} [Sat Aug 6 22:04:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:04:01.825201 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17b5f6a-7856-4bd4-b35c-2fd003a9d6b9 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:04:01.826821 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-a17b5f6a-7856-4bd4-b35c-2fd003a9d6b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:04:01.973752 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17b5f6a-7856-4bd4-b35c-2fd003a9d6b9 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:04:01.999565 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:04:02.087511 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372de0fd-56a4-4b31-8108-c85c6269d6f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a1319d33e9144859c6010843be312a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:04:02.088036 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372de0fd-56a4-4b31-8108-c85c6269d6f5 service nova] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:04:02.088316 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372de0fd-56a4-4b31-8108-c85c6269d6f5 service nova] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:04:02.088615 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372de0fd-56a4-4b31-8108-c85c6269d6f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:04:02.090238 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-372de0fd-56a4-4b31-8108-c85c6269d6f5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:04:02.178686 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372de0fd-56a4-4b31-8108-c85c6269d6f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:04:02.180287 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372de0fd-56a4-4b31-8108-c85c6269d6f5 service nova] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:04:02.198549 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 720/1440] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:04:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:04:04.121659 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d25d110-d215-46c3-b145-fa26f67d7e2e None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:04:04.122225 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d25d110-d215-46c3-b145-fa26f67d7e2e None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:04:04.122225 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d25d110-d215-46c3-b145-fa26f67d7e2e None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:04:04.123530 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 721/1441] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 6 22:04:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:04:04.129088 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e768ae0-6358-46ff-b282-6e7f9da411e2 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:04:04.129541 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e768ae0-6358-46ff-b282-6e7f9da411e2 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:04:04.129640 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e768ae0-6358-46ff-b282-6e7f9da411e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:04:04.170968 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:04:04.196235 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-5e768ae0-6358-46ff-b282-6e7f9da411e2 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:04:04.197485 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-5e768ae0-6358-46ff-b282-6e7f9da411e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:04:04.294734 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 721/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 6 22:04:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:04:04.333284 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d25d110-d215-46c3-b145-fa26f67d7e2e None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:04:04.334800 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-6d25d110-d215-46c3-b145-fa26f67d7e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:04:04.441469 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d25d110-d215-46c3-b145-fa26f67d7e2e None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:04:04.473084 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:04:04.516790 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4731e1a9-3133-48f1-bc73-bbff4f10021b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:04:04.517392 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4731e1a9-3133-48f1-bc73-bbff4f10021b service neutron] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:04:04.517676 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4731e1a9-3133-48f1-bc73-bbff4f10021b service neutron] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:04:04.517948 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4731e1a9-3133-48f1-bc73-bbff4f10021b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:04:04.519545 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-4731e1a9-3133-48f1-bc73-bbff4f10021b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:04:04.627183 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4731e1a9-3133-48f1-bc73-bbff4f10021b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:04:04.628625 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4731e1a9-3133-48f1-bc73-bbff4f10021b service neutron] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:04:04.646738 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 722/1443] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:04:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:04:06.787203 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b827797-bd25-4f4f-8bcf-53af3b6d2fbb None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:04:06.787744 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b827797-bd25-4f4f-8bcf-53af3b6d2fbb None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:04:06.787744 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b827797-bd25-4f4f-8bcf-53af3b6d2fbb None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:04:06.789121 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 722/1444] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 6 22:04:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:04:06.794229 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8091df-8efa-48bb-ba10-518ca0d06d4a None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:04:06.794681 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8091df-8efa-48bb-ba10-518ca0d06d4a None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:04:06.794790 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8091df-8efa-48bb-ba10-518ca0d06d4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:04:06.859836 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-4c8091df-8efa-48bb-ba10-518ca0d06d4a None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:04:06.861739 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-4c8091df-8efa-48bb-ba10-518ca0d06d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:04:06.874392 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:04:06.957465 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 723/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 6 22:04:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:04:06.995499 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b827797-bd25-4f4f-8bcf-53af3b6d2fbb None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:04:06.997120 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-7b827797-bd25-4f4f-8bcf-53af3b6d2fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:04:07.099568 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b827797-bd25-4f4f-8bcf-53af3b6d2fbb None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:04:07.112029 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:04:07.171753 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5247b4-9c52-4557-b76c-3239dd61b43e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:04:07.172358 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5247b4-9c52-4557-b76c-3239dd61b43e service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:04:07.173246 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5247b4-9c52-4557-b76c-3239dd61b43e service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:04:07.173525 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5247b4-9c52-4557-b76c-3239dd61b43e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:04:07.175118 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-4c5247b4-9c52-4557-b76c-3239dd61b43e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:04:07.282781 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5247b4-9c52-4557-b76c-3239dd61b43e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:04:07.284220 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5247b4-9c52-4557-b76c-3239dd61b43e service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:04:07.302049 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 723/1446] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:04:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:04:09.687773 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdb65ec-5285-4779-839b-9577725f2f4f None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:04:09.688888 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdb65ec-5285-4779-839b-9577725f2f4f None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:04:09.688888 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdb65ec-5285-4779-839b-9577725f2f4f None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:04:09.690164 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 724/1447] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 6 22:04:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:04:09.695312 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8534bb5-6d3a-4914-85b4-2613ba0cb59d None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:04:09.695764 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8534bb5-6d3a-4914-85b4-2613ba0cb59d None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:04:09.695873 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8534bb5-6d3a-4914-85b4-2613ba0cb59d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:04:09.746146 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:04:09.766923 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-e8534bb5-6d3a-4914-85b4-2613ba0cb59d None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:04:09.768179 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-e8534bb5-6d3a-4914-85b4-2613ba0cb59d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:04:09.985976 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 724/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 6 22:04:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:04:10.025821 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdb65ec-5285-4779-839b-9577725f2f4f None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:04:10.027358 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-afdb65ec-5285-4779-839b-9577725f2f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:04:10.130869 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdb65ec-5285-4779-839b-9577725f2f4f None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:04:10.170909 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:04:10.203859 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1549f46-512f-4538-bc05-82f293447d95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:04:10.204483 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1549f46-512f-4538-bc05-82f293447d95 service neutron] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:04:10.204762 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1549f46-512f-4538-bc05-82f293447d95 service neutron] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:04:10.205031 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1549f46-512f-4538-bc05-82f293447d95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:04:10.206628 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-a1549f46-512f-4538-bc05-82f293447d95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:04:10.313584 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1549f46-512f-4538-bc05-82f293447d95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:04:10.314920 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1549f46-512f-4538-bc05-82f293447d95 service neutron] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:04:10.333088 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 725/1449] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:04:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:04:12.504317 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4555195-b492-4fcd-9a15-8524a7b983fd None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:04:12.504921 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4555195-b492-4fcd-9a15-8524a7b983fd None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:04:12.504921 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4555195-b492-4fcd-9a15-8524a7b983fd None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:04:12.506225 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 725/1450] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 6 22:04:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:04:12.511324 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff1984c-2502-498f-9728-9e31c327f23b None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:04:12.511763 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff1984c-2502-498f-9728-9e31c327f23b None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:04:12.511866 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff1984c-2502-498f-9728-9e31c327f23b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:04:12.576836 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-7ff1984c-2502-498f-9728-9e31c327f23b None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:04:12.578032 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-7ff1984c-2502-498f-9728-9e31c327f23b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:04:12.598473 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:04:12.675423 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 726/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 6 22:04:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:04:12.713800 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4555195-b492-4fcd-9a15-8524a7b983fd None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:04:12.715305 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-c4555195-b492-4fcd-9a15-8524a7b983fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:04:12.817519 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4555195-b492-4fcd-9a15-8524a7b983fd None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:04:12.837598 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:04:12.894948 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffb4446-3848-4594-b422-949e1449e6b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:04:12.895554 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffb4446-3848-4594-b422-949e1449e6b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:04:12.895831 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffb4446-3848-4594-b422-949e1449e6b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:04:12.896102 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffb4446-3848-4594-b422-949e1449e6b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:04:12.897734 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-bffb4446-3848-4594-b422-949e1449e6b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:04:13.004860 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffb4446-3848-4594-b422-949e1449e6b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:04:13.006306 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffb4446-3848-4594-b422-949e1449e6b5 service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:04:13.024431 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 726/1452] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:04:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:04:15.084578 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2cb7ba-8b6a-4724-abcd-3b3e4135527e None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:04:15.085141 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2cb7ba-8b6a-4724-abcd-3b3e4135527e None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:04:15.085141 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2cb7ba-8b6a-4724-abcd-3b3e4135527e None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:04:15.086472 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 727/1453] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 6 22:04:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:04:15.091618 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80720c39-55b3-495c-8be6-e324c27caef3 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:04:15.092033 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80720c39-55b3-495c-8be6-e324c27caef3 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:04:15.092158 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80720c39-55b3-495c-8be6-e324c27caef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:04:15.157327 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:04:15.159931 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-80720c39-55b3-495c-8be6-e324c27caef3 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:04:15.161169 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-80720c39-55b3-495c-8be6-e324c27caef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:04:15.256323 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 727/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 6 22:04:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:04:15.294843 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2cb7ba-8b6a-4724-abcd-3b3e4135527e None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:04:15.296417 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-8e2cb7ba-8b6a-4724-abcd-3b3e4135527e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:04:15.400335 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2cb7ba-8b6a-4724-abcd-3b3e4135527e None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:04:15.447836 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:04:15.472361 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819c50ed-0658-4463-af99-b73da577aced service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:04:15.472986 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819c50ed-0658-4463-af99-b73da577aced service neutron] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:04:15.473257 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819c50ed-0658-4463-af99-b73da577aced service neutron] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:04:15.473521 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819c50ed-0658-4463-af99-b73da577aced service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:04:15.475105 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-819c50ed-0658-4463-af99-b73da577aced service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:04:15.584719 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819c50ed-0658-4463-af99-b73da577aced service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:04:15.586134 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819c50ed-0658-4463-af99-b73da577aced service neutron] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:04:15.604417 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 728/1455] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:04:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:04:17.691297 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09151e55-c401-4edd-9bbf-20f6dfc0a43e None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:04:17.691851 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09151e55-c401-4edd-9bbf-20f6dfc0a43e None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:04:17.691851 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09151e55-c401-4edd-9bbf-20f6dfc0a43e None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:04:17.693101 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 728/1456] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 6 22:04:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:04:17.698254 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcb821b-c4a1-4e63-8bc9-1a1f4bba14f6 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:04:17.698681 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcb821b-c4a1-4e63-8bc9-1a1f4bba14f6 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:04:17.698786 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcb821b-c4a1-4e63-8bc9-1a1f4bba14f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:04:17.766033 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-cbcb821b-c4a1-4e63-8bc9-1a1f4bba14f6 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:04:17.767230 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-cbcb821b-c4a1-4e63-8bc9-1a1f4bba14f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:04:17.796773 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:04:17.865722 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 729/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 6 22:04:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 22:04:17.903247 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09151e55-c401-4edd-9bbf-20f6dfc0a43e None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:04:17.904783 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-09151e55-c401-4edd-9bbf-20f6dfc0a43e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:04:18.008789 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09151e55-c401-4edd-9bbf-20f6dfc0a43e None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:04:18.036167 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:04:18.081514 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1775f0-5956-468b-a2ca-a590ca8d9e3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:04:18.082116 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1775f0-5956-468b-a2ca-a590ca8d9e3b service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:04:18.082396 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1775f0-5956-468b-a2ca-a590ca8d9e3b service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:04:18.082666 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1775f0-5956-468b-a2ca-a590ca8d9e3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:04:18.084263 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-1a1775f0-5956-468b-a2ca-a590ca8d9e3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:04:18.192673 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1775f0-5956-468b-a2ca-a590ca8d9e3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:04:18.194104 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1775f0-5956-468b-a2ca-a590ca8d9e3b service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:04:18.212520 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 729/1458] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:04:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:04:20.270389 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687f84d9-4937-432f-a823-31e10d8ecd51 None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:04:20.270984 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687f84d9-4937-432f-a823-31e10d8ecd51 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:04:20.270984 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687f84d9-4937-432f-a823-31e10d8ecd51 None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:04:20.272369 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 730/1459] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 6 22:04:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:04:20.277490 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d23f51-86ef-40bf-9cad-ac835474f4a4 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:04:20.277926 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d23f51-86ef-40bf-9cad-ac835474f4a4 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:04:20.278034 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d23f51-86ef-40bf-9cad-ac835474f4a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:04:20.342460 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-b3d23f51-86ef-40bf-9cad-ac835474f4a4 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:04:20.343744 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-b3d23f51-86ef-40bf-9cad-ac835474f4a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:04:20.352379 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:04:20.441907 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 730/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 6 22:04:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:04:20.479851 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687f84d9-4937-432f-a823-31e10d8ecd51 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:04:20.481451 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-687f84d9-4937-432f-a823-31e10d8ecd51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:04:20.583710 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687f84d9-4937-432f-a823-31e10d8ecd51 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:04:20.638799 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:04:20.657990 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1cde41-a2ab-4338-9748-87f2b7264878 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:04:20.658590 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1cde41-a2ab-4338-9748-87f2b7264878 service neutron] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:04:20.658864 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1cde41-a2ab-4338-9748-87f2b7264878 service neutron] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:04:20.659182 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1cde41-a2ab-4338-9748-87f2b7264878 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:04:20.660795 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-cc1cde41-a2ab-4338-9748-87f2b7264878 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:04:20.767893 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1cde41-a2ab-4338-9748-87f2b7264878 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:04:20.769344 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1cde41-a2ab-4338-9748-87f2b7264878 service neutron] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:04:20.788104 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 731/1461] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:04:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:04:22.862599 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7fc24c-8fff-4618-b799-699f799f136a None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:04:22.863142 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7fc24c-8fff-4618-b799-699f799f136a None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:04:22.863142 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7fc24c-8fff-4618-b799-699f799f136a None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:04:22.864478 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 731/1462] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 6 22:04:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:04:22.869886 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc1b2d5-2ab2-41b4-a0fa-e8d603c9790c None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:04:22.870322 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc1b2d5-2ab2-41b4-a0fa-e8d603c9790c None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:04:22.870421 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc1b2d5-2ab2-41b4-a0fa-e8d603c9790c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:04:22.936079 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-8dc1b2d5-2ab2-41b4-a0fa-e8d603c9790c None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:04:22.937302 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-8dc1b2d5-2ab2-41b4-a0fa-e8d603c9790c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:04:22.975094 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:04:23.033274 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 732/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 6 22:04:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:04:23.071132 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7fc24c-8fff-4618-b799-699f799f136a None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:04:23.072669 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-ac7fc24c-8fff-4618-b799-699f799f136a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:04:23.174300 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7fc24c-8fff-4618-b799-699f799f136a None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:04:23.210819 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:04:23.247527 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883c8f85-5987-43b6-b06d-b7ef19b4f200 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:04:23.248124 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883c8f85-5987-43b6-b06d-b7ef19b4f200 service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:04:23.248395 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883c8f85-5987-43b6-b06d-b7ef19b4f200 service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:04:23.248683 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883c8f85-5987-43b6-b06d-b7ef19b4f200 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:04:23.250243 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-883c8f85-5987-43b6-b06d-b7ef19b4f200 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:04:23.356133 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883c8f85-5987-43b6-b06d-b7ef19b4f200 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:04:23.357604 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883c8f85-5987-43b6-b06d-b7ef19b4f200 service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:04:23.375811 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 732/1464] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:04:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:04:25.446356 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73c05ff-e97d-441f-b06f-f922c5c2ed10 None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:04:25.446920 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73c05ff-e97d-441f-b06f-f922c5c2ed10 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:04:25.446920 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73c05ff-e97d-441f-b06f-f922c5c2ed10 None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:04:25.448294 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 733/1465] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 6 22:04:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:04:25.453978 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa1ed8-f914-4de3-bf4b-6b74687a11eb None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:04:25.454418 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa1ed8-f914-4de3-bf4b-6b74687a11eb None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:04:25.454538 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa1ed8-f914-4de3-bf4b-6b74687a11eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:04:25.521882 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-2aaa1ed8-f914-4de3-bf4b-6b74687a11eb None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:04:25.523014 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-2aaa1ed8-f914-4de3-bf4b-6b74687a11eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:04:25.541693 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:04:25.620282 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 733/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 6 22:04:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:04:25.658670 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73c05ff-e97d-441f-b06f-f922c5c2ed10 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:04:25.660189 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-e73c05ff-e97d-441f-b06f-f922c5c2ed10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:04:25.764773 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73c05ff-e97d-441f-b06f-f922c5c2ed10 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:04:25.831907 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:04:25.838780 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95254996-e212-44a9-bc71-6daff891f260 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:04:25.842346 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95254996-e212-44a9-bc71-6daff891f260 service neutron] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:04:25.842346 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95254996-e212-44a9-bc71-6daff891f260 service neutron] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:04:25.842346 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95254996-e212-44a9-bc71-6daff891f260 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:04:25.842346 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-95254996-e212-44a9-bc71-6daff891f260 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:04:25.949172 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95254996-e212-44a9-bc71-6daff891f260 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:04:25.950588 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95254996-e212-44a9-bc71-6daff891f260 service neutron] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:04:25.969298 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 734/1467] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:04:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:04:27.995795 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70eb853-7d04-4844-a33b-7131df4c419b None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:04:27.996353 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70eb853-7d04-4844-a33b-7131df4c419b None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:04:27.996353 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70eb853-7d04-4844-a33b-7131df4c419b None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:04:27.997740 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 734/1468] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 6 22:04:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:04:28.003885 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6231234-3217-4c63-97ed-6017a7adc764 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:04:28.004309 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6231234-3217-4c63-97ed-6017a7adc764 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:04:28.004441 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6231234-3217-4c63-97ed-6017a7adc764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:04:28.069498 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-b6231234-3217-4c63-97ed-6017a7adc764 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:04:28.070691 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-b6231234-3217-4c63-97ed-6017a7adc764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:04:28.118781 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:04:28.167224 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 735/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 6 22:04:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:04:28.203908 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70eb853-7d04-4844-a33b-7131df4c419b None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:04:28.205444 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-b70eb853-7d04-4844-a33b-7131df4c419b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:04:28.307672 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70eb853-7d04-4844-a33b-7131df4c419b None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:04:28.354924 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:04:28.381790 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880ee429-f826-4cca-a29f-98aca7d705b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:04:28.382385 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880ee429-f826-4cca-a29f-98aca7d705b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:04:28.382658 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880ee429-f826-4cca-a29f-98aca7d705b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:04:28.382920 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880ee429-f826-4cca-a29f-98aca7d705b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:04:28.384530 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-880ee429-f826-4cca-a29f-98aca7d705b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:04:28.490966 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880ee429-f826-4cca-a29f-98aca7d705b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:04:28.492393 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880ee429-f826-4cca-a29f-98aca7d705b2 service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:04:28.510185 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 735/1470] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:04:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 22:04:38.165457 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=22 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:04:38.507961 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=66 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:04:48.106781 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc68566c-fc6b-4e5f-9008-c27c0220e166 None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:04:48.107364 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc68566c-fc6b-4e5f-9008-c27c0220e166 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:04:48.107364 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc68566c-fc6b-4e5f-9008-c27c0220e166 None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:04:48.108875 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 736/1471] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 22:04:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:04:48.114205 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b0f17-2522-4722-bd84-2364ede090f1 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:04:48.114625 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b0f17-2522-4722-bd84-2364ede090f1 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:04:48.114739 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b0f17-2522-4722-bd84-2364ede090f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:04:48.179725 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-f42b0f17-2522-4722-bd84-2364ede090f1 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:04:48.180968 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-f42b0f17-2522-4722-bd84-2364ede090f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:04:48.276125 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 736/1472] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 22:04:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:04:48.364822 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b9ad4b-c6a7-4c74-9b46-e433479cc6af None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:04:48.365104 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b9ad4b-c6a7-4c74-9b46-e433479cc6af None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:04:48.365404 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b9ad4b-c6a7-4c74-9b46-e433479cc6af None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:04:48.366764 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 737/1473] 10.222.0.18 () {54 vars in 966 bytes} [Sat Aug 6 22:04:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:04:48.371441 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec40f721-2570-437c-aae2-270a5cdd9526 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:04:48.371724 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec40f721-2570-437c-aae2-270a5cdd9526 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:04:48.371995 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec40f721-2570-437c-aae2-270a5cdd9526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:04:48.447754 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-ec40f721-2570-437c-aae2-270a5cdd9526 None None] MFA Rules not processed for user `c8d38e73e105476e8de858cfa939ba20`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:04:48.448983 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-ec40f721-2570-437c-aae2-270a5cdd9526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:04:48.469677 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:04:48.524558 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 737/1474] 10.222.0.18 () {58 vars in 1063 bytes} [Sat Aug 6 22:04:48 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:04:48.531979 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b9ad4b-c6a7-4c74-9b46-e433479cc6af None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:04:48.533545 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-52b9ad4b-c6a7-4c74-9b46-e433479cc6af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:04:48.636962 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b9ad4b-c6a7-4c74-9b46-e433479cc6af None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:04:48.711419 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d62667d-b4e2-4059-a798-33638ad8a185 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8d38e73e105476e8de858cfa939ba20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:04:48.712020 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d62667d-b4e2-4059-a798-33638ad8a185 service glance] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:04:48.712298 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d62667d-b4e2-4059-a798-33638ad8a185 service glance] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:04:48.712601 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d62667d-b4e2-4059-a798-33638ad8a185 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:04:48.714192 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-0d62667d-b4e2-4059-a798-33638ad8a185 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:04:48.769242 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:04:48.824273 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d62667d-b4e2-4059-a798-33638ad8a185 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:04:48.825737 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d62667d-b4e2-4059-a798-33638ad8a185 service glance] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:04:48.843761 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 738/1475] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:04:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:04:50.917317 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d323ef-ca10-4e6b-9b6d-b093db35e62e None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:04:50.917873 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d323ef-ca10-4e6b-9b6d-b093db35e62e None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:04:50.917873 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d323ef-ca10-4e6b-9b6d-b093db35e62e None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:04:50.919218 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 738/1476] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 22:04:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:04:50.925062 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f28aa9b-f34f-4b8b-af4a-6232b4b036a6 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:04:50.925496 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f28aa9b-f34f-4b8b-af4a-6232b4b036a6 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:04:50.925625 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f28aa9b-f34f-4b8b-af4a-6232b4b036a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:04:50.990710 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-5f28aa9b-f34f-4b8b-af4a-6232b4b036a6 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:04:50.991809 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-5f28aa9b-f34f-4b8b-af4a-6232b4b036a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:04:51.085923 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 739/1477] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 22:04:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:04:51.109458 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d323ef-ca10-4e6b-9b6d-b093db35e62e None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:04:51.110954 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-a9d323ef-ca10-4e6b-9b6d-b093db35e62e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:04:51.234452 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d323ef-ca10-4e6b-9b6d-b093db35e62e None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:04:51.282339 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:04:51.325319 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9afa21-10b1-4a9c-9417-6eb6bdd32581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:04:51.325910 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9afa21-10b1-4a9c-9417-6eb6bdd32581 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:04:51.326179 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9afa21-10b1-4a9c-9417-6eb6bdd32581 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:04:51.327113 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9afa21-10b1-4a9c-9417-6eb6bdd32581 admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:04:51.327833 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9afa21-10b1-4a9c-9417-6eb6bdd32581 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:04:51.329583 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: WARNING py.warnings [None req-dc9afa21-10b1-4a9c-9417-6eb6bdd32581 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 06 22:04:51.329583 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: warnings.warn(msg) Aug 06 22:04:51.329583 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: Aug 06 22:04:51.330705 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9afa21-10b1-4a9c-9417-6eb6bdd32581 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:04:51.349552 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 739/1478] 10.222.0.18 () {56 vars in 1108 bytes} [Sat Aug 6 22:04:51 2022] GET /identity/v3/projects => generated 2553 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:04:53.364990 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a385377-ef8b-43dc-8a82-ce6da29950b4 None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:04:53.365475 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a385377-ef8b-43dc-8a82-ce6da29950b4 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:04:53.365574 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a385377-ef8b-43dc-8a82-ce6da29950b4 None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:04:53.366866 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 740/1479] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 22:04:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:04:53.372276 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac77a1d-6c7a-4502-9bfa-32af8f4095c3 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:04:53.372718 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac77a1d-6c7a-4502-9bfa-32af8f4095c3 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:04:53.372878 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac77a1d-6c7a-4502-9bfa-32af8f4095c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:04:53.438443 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-9ac77a1d-6c7a-4502-9bfa-32af8f4095c3 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:04:53.439657 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-9ac77a1d-6c7a-4502-9bfa-32af8f4095c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:04:53.536256 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 740/1480] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 22:04:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:04:53.578895 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a385377-ef8b-43dc-8a82-ce6da29950b4 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:04:53.580475 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-0a385377-ef8b-43dc-8a82-ce6da29950b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:04:53.590218 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:04:53.725610 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a385377-ef8b-43dc-8a82-ce6da29950b4 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:04:53.805014 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:04:53.839688 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d14e5c-2e54-4865-ae3d-9e4d83f5829d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a1319d33e9144859c6010843be312a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:04:53.840293 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d14e5c-2e54-4865-ae3d-9e4d83f5829d service nova] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:04:53.840605 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d14e5c-2e54-4865-ae3d-9e4d83f5829d service nova] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:04:53.840881 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d14e5c-2e54-4865-ae3d-9e4d83f5829d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:04:53.842524 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-06d14e5c-2e54-4865-ae3d-9e4d83f5829d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:04:53.929653 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d14e5c-2e54-4865-ae3d-9e4d83f5829d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:04:53.931218 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d14e5c-2e54-4865-ae3d-9e4d83f5829d service nova] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:04:53.949220 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 741/1481] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:04:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:04:56.121891 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d39b9-4100-4530-8d9d-56161dbc83c4 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:04:56.122494 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d39b9-4100-4530-8d9d-56161dbc83c4 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:04:56.122494 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d39b9-4100-4530-8d9d-56161dbc83c4 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:04:56.124382 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 741/1482] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 22:04:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:04:56.129797 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f734d-263b-4de8-9b8a-fe4021653d7c None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:04:56.130202 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f734d-263b-4de8-9b8a-fe4021653d7c None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:04:56.130334 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f734d-263b-4de8-9b8a-fe4021653d7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:04:56.197017 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-719f734d-263b-4de8-9b8a-fe4021653d7c None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:04:56.198424 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-719f734d-263b-4de8-9b8a-fe4021653d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:04:56.245808 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:04:56.295596 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 742/1483] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 22:04:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:04:56.332165 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7d39b9-4100-4530-8d9d-56161dbc83c4 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:04:56.335901 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-ad7d39b9-4100-4530-8d9d-56161dbc83c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:04:56.359298 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:04:56.441020 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7d39b9-4100-4530-8d9d-56161dbc83c4 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:04:56.514697 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd74f0db-9e09-4191-8458-563e79ec5f4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:04:56.515305 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd74f0db-9e09-4191-8458-563e79ec5f4c service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:04:56.515584 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd74f0db-9e09-4191-8458-563e79ec5f4c service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:04:56.515854 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd74f0db-9e09-4191-8458-563e79ec5f4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:04:56.517473 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-dd74f0db-9e09-4191-8458-563e79ec5f4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:04:56.591409 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:04:56.626873 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd74f0db-9e09-4191-8458-563e79ec5f4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:04:56.628647 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd74f0db-9e09-4191-8458-563e79ec5f4c service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:04:56.646576 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 742/1484] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:04:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:04:58.704283 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b27543-ea15-4bd2-996f-1ad2d1070c2e None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:04:58.704855 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b27543-ea15-4bd2-996f-1ad2d1070c2e None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:04:58.704855 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b27543-ea15-4bd2-996f-1ad2d1070c2e None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:04:58.706129 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 743/1485] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 22:04:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:04:58.711462 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d85c6b-b892-4cb7-b337-bd3fb8cb465e None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:04:58.713358 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d85c6b-b892-4cb7-b337-bd3fb8cb465e None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:04:58.713358 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d85c6b-b892-4cb7-b337-bd3fb8cb465e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:04:58.777775 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-33d85c6b-b892-4cb7-b337-bd3fb8cb465e None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:04:58.778974 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-33d85c6b-b892-4cb7-b337-bd3fb8cb465e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:04:58.875085 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 743/1486] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 22:04:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:04:58.913514 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b27543-ea15-4bd2-996f-1ad2d1070c2e None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:04:58.915152 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-92b27543-ea15-4bd2-996f-1ad2d1070c2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:04:59.018511 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b27543-ea15-4bd2-996f-1ad2d1070c2e None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:04:59.090216 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:04:59.091987 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dab3bad-e25a-4b84-b130-0ffaab945b78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:04:59.092615 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dab3bad-e25a-4b84-b130-0ffaab945b78 service neutron] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:04:59.092892 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dab3bad-e25a-4b84-b130-0ffaab945b78 service neutron] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:04:59.093164 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dab3bad-e25a-4b84-b130-0ffaab945b78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:04:59.094796 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-0dab3bad-e25a-4b84-b130-0ffaab945b78 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:04:59.201856 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dab3bad-e25a-4b84-b130-0ffaab945b78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:04:59.203358 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dab3bad-e25a-4b84-b130-0ffaab945b78 service neutron] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:04:59.221973 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 744/1487] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:04:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:05:01.305974 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be65b786-803b-433c-acaf-167849180366 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:05:01.306523 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be65b786-803b-433c-acaf-167849180366 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:05:01.306523 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be65b786-803b-433c-acaf-167849180366 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:05:01.307857 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 744/1488] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 22:05:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:05:01.313403 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722183bd-79dd-4eed-ae77-f4d6fa55cbe6 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:05:01.313973 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722183bd-79dd-4eed-ae77-f4d6fa55cbe6 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:05:01.313973 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722183bd-79dd-4eed-ae77-f4d6fa55cbe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:05:01.381300 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-722183bd-79dd-4eed-ae77-f4d6fa55cbe6 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:05:01.382503 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-722183bd-79dd-4eed-ae77-f4d6fa55cbe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:05:01.433766 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:05:01.477585 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 745/1489] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 22:05:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:05:01.519028 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be65b786-803b-433c-acaf-167849180366 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:05:01.521255 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-be65b786-803b-433c-acaf-167849180366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:05:01.559105 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:05:01.626716 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be65b786-803b-433c-acaf-167849180366 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:05:01.700129 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906879ba-48bd-4077-86f5-6f3ff81250d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:05:01.700829 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906879ba-48bd-4077-86f5-6f3ff81250d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:05:01.701106 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906879ba-48bd-4077-86f5-6f3ff81250d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:05:01.701379 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906879ba-48bd-4077-86f5-6f3ff81250d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:05:01.703449 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-906879ba-48bd-4077-86f5-6f3ff81250d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:05:01.785885 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:05:01.815493 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906879ba-48bd-4077-86f5-6f3ff81250d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:05:01.816970 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906879ba-48bd-4077-86f5-6f3ff81250d3 service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:05:01.834223 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 745/1490] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:05:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:05:04.302930 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbeaaa0-8e16-4726-9f2b-d271a3aa482d None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:05:04.303500 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbeaaa0-8e16-4726-9f2b-d271a3aa482d None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:05:04.303500 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbeaaa0-8e16-4726-9f2b-d271a3aa482d None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:05:04.304845 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 746/1491] 10.222.0.18 () {54 vars in 927 bytes} [Sat Aug 6 22:05:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:05:04.310851 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe811f8-b622-4158-bb0a-fd5917951f17 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:05:04.311290 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe811f8-b622-4158-bb0a-fd5917951f17 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:05:04.311409 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe811f8-b622-4158-bb0a-fd5917951f17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:05:04.378202 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-4fe811f8-b622-4158-bb0a-fd5917951f17 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:05:04.379407 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-4fe811f8-b622-4158-bb0a-fd5917951f17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:05:04.474472 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 746/1492] 10.222.0.18 () {58 vars in 1024 bytes} [Sat Aug 6 22:05:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:05:04.514107 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbeaaa0-8e16-4726-9f2b-d271a3aa482d None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:05:04.515634 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-5fbeaaa0-8e16-4726-9f2b-d271a3aa482d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:05:04.617107 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbeaaa0-8e16-4726-9f2b-d271a3aa482d None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:05:04.687885 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1527ec46-f200-4ddd-9909-9c025821547f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:05:04.688505 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1527ec46-f200-4ddd-9909-9c025821547f service neutron] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:05:04.688780 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1527ec46-f200-4ddd-9909-9c025821547f service neutron] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:05:04.689047 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1527ec46-f200-4ddd-9909-9c025821547f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:05:04.690626 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-1527ec46-f200-4ddd-9909-9c025821547f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:05:04.699389 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:05:04.800105 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1527ec46-f200-4ddd-9909-9c025821547f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:05:04.801552 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1527ec46-f200-4ddd-9909-9c025821547f service neutron] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:05:04.819259 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 747/1493] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:05:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:05:14.472688 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=82 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:05:14.816869 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=50 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:06:53.692074 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fedf3ba-ea8b-4c63-aa52-f8a6d22060c5 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:06:53.692673 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fedf3ba-ea8b-4c63-aa52-f8a6d22060c5 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:06:53.692673 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fedf3ba-ea8b-4c63-aa52-f8a6d22060c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:06:53.768227 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-6fedf3ba-ea8b-4c63-aa52-f8a6d22060c5 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:06:53.769445 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-6fedf3ba-ea8b-4c63-aa52-f8a6d22060c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:06:53.864431 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 747/1494] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:06:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:06:53.873207 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7c1d62-3219-4209-8d48-9b99697e17c6 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:06:53.873697 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7c1d62-3219-4209-8d48-9b99697e17c6 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:06:53.873799 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7c1d62-3219-4209-8d48-9b99697e17c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:06:53.947466 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-ea7c1d62-3219-4209-8d48-9b99697e17c6 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:06:53.948612 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-ea7c1d62-3219-4209-8d48-9b99697e17c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:06:54.043651 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 748/1495] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:06:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:06:54.048320 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fedf3ba-ea8b-4c63-aa52-f8a6d22060c5 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:06:54.049736 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-6fedf3ba-ea8b-4c63-aa52-f8a6d22060c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:06:54.061497 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fedf3ba-ea8b-4c63-aa52-f8a6d22060c5 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:06:54.122137 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:06:54.155194 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a906f1a-ef03-481f-a5ec-35b2a13e5416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:06:54.155793 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a906f1a-ef03-481f-a5ec-35b2a13e5416 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:06:54.156070 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a906f1a-ef03-481f-a5ec-35b2a13e5416 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:06:54.156337 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a906f1a-ef03-481f-a5ec-35b2a13e5416 admin admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:06:54.157012 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a906f1a-ef03-481f-a5ec-35b2a13e5416 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 22:06:54.157548 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a906f1a-ef03-481f-a5ec-35b2a13e5416 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:06:54.158853 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: WARNING py.warnings [None req-0a906f1a-ef03-481f-a5ec-35b2a13e5416 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 06 22:06:54.158853 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: warnings.warn(msg) Aug 06 22:06:54.158853 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: Aug 06 22:06:54.160042 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a906f1a-ef03-481f-a5ec-35b2a13e5416 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:06:54.171043 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 748/1496] 10.222.0.18 () {58 vars in 1153 bytes} [Sat Aug 6 22:06:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 22:06:54.179264 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7c1d62-3219-4209-8d48-9b99697e17c6 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:06:54.188855 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7c1d62-3219-4209-8d48-9b99697e17c6 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:06:54.249133 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:06:54.282817 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7b7cbf-0999-4b3d-a1dd-34ff1bdb4c52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:06:54.283416 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7b7cbf-0999-4b3d-a1dd-34ff1bdb4c52 admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:06:54.283692 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7b7cbf-0999-4b3d-a1dd-34ff1bdb4c52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:06:54.283973 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7b7cbf-0999-4b3d-a1dd-34ff1bdb4c52 admin admin] PATH_INFO: `/v3/projects` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:06:54.284606 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7b7cbf-0999-4b3d-a1dd-34ff1bdb4c52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:06:54.286385 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: WARNING py.warnings [None req-3b7b7cbf-0999-4b3d-a1dd-34ff1bdb4c52 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 06 22:06:54.286385 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: warnings.warn(msg) Aug 06 22:06:54.286385 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: Aug 06 22:06:54.287574 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7b7cbf-0999-4b3d-a1dd-34ff1bdb4c52 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:06:54.421292 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 749/1497] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:06:54 2022] POST /identity/v3/projects => generated 379 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:06:54.425628 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a906f1a-ef03-481f-a5ec-35b2a13e5416 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:06:54.427163 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-0a906f1a-ef03-481f-a5ec-35b2a13e5416 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:06:54.459975 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a906f1a-ef03-481f-a5ec-35b2a13e5416 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:06:54.553218 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41614af-0c5d-482c-8b5b-ace4c1c2c48e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:06:54.553736 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41614af-0c5d-482c-8b5b-ace4c1c2c48e admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:06:54.554007 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41614af-0c5d-482c-8b5b-ace4c1c2c48e admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:06:54.554269 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41614af-0c5d-482c-8b5b-ace4c1c2c48e admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:06:54.555561 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41614af-0c5d-482c-8b5b-ace4c1c2c48e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:06:54.556650 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: WARNING py.warnings [None req-a41614af-0c5d-482c-8b5b-ace4c1c2c48e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 06 22:06:54.556650 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: warnings.warn(msg) Aug 06 22:06:54.556650 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: Aug 06 22:06:54.557846 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41614af-0c5d-482c-8b5b-ace4c1c2c48e admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:06:54.647749 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 749/1498] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:06:54 2022] POST /identity/v3/users => generated 290 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 22:06:54.651960 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7b7cbf-0999-4b3d-a1dd-34ff1bdb4c52 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:06:54.653481 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-3b7b7cbf-0999-4b3d-a1dd-34ff1bdb4c52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:06:54.714496 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7b7cbf-0999-4b3d-a1dd-34ff1bdb4c52 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:06:54.798578 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:06:54.809217 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a49ace-217e-479e-9b74-6034ec885779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:06:54.809793 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a49ace-217e-479e-9b74-6034ec885779 admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:06:54.810071 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a49ace-217e-479e-9b74-6034ec885779 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:06:54.810337 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a49ace-217e-479e-9b74-6034ec885779 admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:06:54.810977 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a49ace-217e-479e-9b74-6034ec885779 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:06:54.811977 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: WARNING py.warnings [None req-24a49ace-217e-479e-9b74-6034ec885779 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 06 22:06:54.811977 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: warnings.warn(msg) Aug 06 22:06:54.811977 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: Aug 06 22:06:54.812886 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a49ace-217e-479e-9b74-6034ec885779 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:06:54.824992 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 750/1499] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:06:54 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:06:54.829101 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41614af-0c5d-482c-8b5b-ace4c1c2c48e admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:06:54.830651 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-a41614af-0c5d-482c-8b5b-ace4c1c2c48e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:06:54.842616 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41614af-0c5d-482c-8b5b-ace4c1c2c48e admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:06:54.867952 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:06:54.937797 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131dc612-87f8-46ae-961f-641d4b831fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:06:54.938395 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131dc612-87f8-46ae-961f-641d4b831fc2 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:06:54.938669 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131dc612-87f8-46ae-961f-641d4b831fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:06:54.938938 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131dc612-87f8-46ae-961f-641d4b831fc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:06:54.939574 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131dc612-87f8-46ae-961f-641d4b831fc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:06:54.940626 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: WARNING py.warnings [None req-131dc612-87f8-46ae-961f-641d4b831fc2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 06 22:06:54.940626 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: warnings.warn(msg) Aug 06 22:06:54.940626 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: Aug 06 22:06:54.941520 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131dc612-87f8-46ae-961f-641d4b831fc2 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:06:54.953698 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 750/1500] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:06:54 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:06:54.957858 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a49ace-217e-479e-9b74-6034ec885779 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:06:54.959363 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-24a49ace-217e-479e-9b74-6034ec885779 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:06:54.994448 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a49ace-217e-479e-9b74-6034ec885779 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:06:55.084096 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6291223e-7cbe-4438-8f4f-0ab05ab3ebab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:06:55.084704 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6291223e-7cbe-4438-8f4f-0ab05ab3ebab admin admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:06:55.084979 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6291223e-7cbe-4438-8f4f-0ab05ab3ebab admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:06:55.085242 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6291223e-7cbe-4438-8f4f-0ab05ab3ebab admin admin] PATH_INFO: `/v3/projects/ab4494e9106c4752baebdd66ce0e2b5b/users/989f256843c54a8f973d173ab6184175/roles/56e4a96094e9434cbbafb22579c034f5` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:06:55.108166 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6291223e-7cbe-4438-8f4f-0ab05ab3ebab admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab4494e9106c4752baebdd66ce0e2b5b, user_id=989f256843c54a8f973d173ab6184175, role_id=56e4a96094e9434cbbafb22579c034f5)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:06:55.109219 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: WARNING py.warnings [None req-6291223e-7cbe-4438-8f4f-0ab05ab3ebab admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 06 22:06:55.109219 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: warnings.warn(msg) Aug 06 22:06:55.109219 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: Aug 06 22:06:55.111149 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6291223e-7cbe-4438-8f4f-0ab05ab3ebab admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:06:55.210046 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 751/1501] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:06:54 2022] PUT /identity/v3/projects/ab4494e9106c4752baebdd66ce0e2b5b/users/989f256843c54a8f973d173ab6184175/roles/56e4a96094e9434cbbafb22579c034f5 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:06:55.222947 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36c8a34-028f-4e44-a100-acc3fa51bc29 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:06:55.223231 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36c8a34-028f-4e44-a100-acc3fa51bc29 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:06:55.223534 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36c8a34-028f-4e44-a100-acc3fa51bc29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:06:55.281821 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-f36c8a34-028f-4e44-a100-acc3fa51bc29 None None] MFA Rules not processed for user `989f256843c54a8f973d173ab6184175`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:06:55.283035 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-f36c8a34-028f-4e44-a100-acc3fa51bc29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:06:55.360629 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 751/1502] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:06:55 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:06:55.373611 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1d8a4f-2cb8-455f-9411-e3a62fd55e72 None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:06:55.373919 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1d8a4f-2cb8-455f-9411-e3a62fd55e72 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:06:55.374193 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1d8a4f-2cb8-455f-9411-e3a62fd55e72 None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:06:55.375576 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 752/1503] 10.222.0.18 () {54 vars in 964 bytes} [Sat Aug 6 22:06:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:06:55.380371 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32878ae0-c642-4520-ba33-6a96043fc8b0 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:06:55.380688 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32878ae0-c642-4520-ba33-6a96043fc8b0 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:06:55.380963 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32878ae0-c642-4520-ba33-6a96043fc8b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:06:55.412620 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:06:55.456856 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-32878ae0-c642-4520-ba33-6a96043fc8b0 None None] MFA Rules not processed for user `aa71c71de263478da234f3c5716a6e5f`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:06:55.458056 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-32878ae0-c642-4520-ba33-6a96043fc8b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:06:55.532848 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 752/1504] 10.222.0.18 () {58 vars in 1061 bytes} [Sat Aug 6 22:06:55 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:06:55.540651 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1d8a4f-2cb8-455f-9411-e3a62fd55e72 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:06:55.542185 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-dd1d8a4f-2cb8-455f-9411-e3a62fd55e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:06:55.591914 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:06:55.645143 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1d8a4f-2cb8-455f-9411-e3a62fd55e72 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:06:55.717934 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb58466-1025-4611-af73-0748b1cd8627 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa71c71de263478da234f3c5716a6e5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:06:55.718521 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb58466-1025-4611-af73-0748b1cd8627 service cinder] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:06:55.718794 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb58466-1025-4611-af73-0748b1cd8627 service cinder] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:06:55.719060 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb58466-1025-4611-af73-0748b1cd8627 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:06:55.720671 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-dfb58466-1025-4611-af73-0748b1cd8627 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:06:55.816720 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb58466-1025-4611-af73-0748b1cd8627 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:06:55.818136 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb58466-1025-4611-af73-0748b1cd8627 service cinder] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:06:55.822154 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:06:55.836470 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 753/1505] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:06:55 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:06:55.881430 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c49724-a3d2-490e-a099-66d4f48e1bae None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:06:55.881828 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c49724-a3d2-490e-a099-66d4f48e1bae None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:06:55.881975 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c49724-a3d2-490e-a099-66d4f48e1bae None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:06:55.883834 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 753/1506] 10.222.0.18 () {52 vars in 830 bytes} [Sat Aug 6 22:06:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:06:55.887607 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb58466-1025-4611-af73-0748b1cd8627 service cinder] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:06:55.888909 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-dfb58466-1025-4611-af73-0748b1cd8627 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:06:55.989438 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb58466-1025-4611-af73-0748b1cd8627 service cinder] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:06:56.083231 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931e4e79-4bd2-4f52-a2d7-1d872b3310a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:06:56.083859 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931e4e79-4bd2-4f52-a2d7-1d872b3310a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:06:56.084137 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931e4e79-4bd2-4f52-a2d7-1d872b3310a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:06:56.084435 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931e4e79-4bd2-4f52-a2d7-1d872b3310a2 admin admin] PATH_INFO: `/v3/users/989f256843c54a8f973d173ab6184175` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:06:56.096561 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931e4e79-4bd2-4f52-a2d7-1d872b3310a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=989f256843c54a8f973d173ab6184175)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:06:56.097574 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: WARNING py.warnings [None req-931e4e79-4bd2-4f52-a2d7-1d872b3310a2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 06 22:06:56.097574 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: warnings.warn(msg) Aug 06 22:06:56.097574 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: Aug 06 22:06:56.098800 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931e4e79-4bd2-4f52-a2d7-1d872b3310a2 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:06:56.109580 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:06:56.310297 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-931e4e79-4bd2-4f52-a2d7-1d872b3310a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '989f256843c54a8f973d173ab6184175'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:06:56.332855 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-931e4e79-4bd2-4f52-a2d7-1d872b3310a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '989f256843c54a8f973d173ab6184175'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:06:56.409196 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-931e4e79-4bd2-4f52-a2d7-1d872b3310a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '989f256843c54a8f973d173ab6184175'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:06:56.425342 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-931e4e79-4bd2-4f52-a2d7-1d872b3310a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '989f256843c54a8f973d173ab6184175'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:06:56.425796 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-931e4e79-4bd2-4f52-a2d7-1d872b3310a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '989f256843c54a8f973d173ab6184175'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:06:56.431700 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 754/1507] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:06:55 2022] DELETE /identity/v3/users/989f256843c54a8f973d173ab6184175 => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:06:56.436253 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e23b01-ecc0-4298-b713-3f0756dcd15e None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:06:56.436604 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e23b01-ecc0-4298-b713-3f0756dcd15e None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:06:56.436880 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e23b01-ecc0-4298-b713-3f0756dcd15e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:06:56.498200 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-f6e23b01-ecc0-4298-b713-3f0756dcd15e None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:06:56.499396 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-f6e23b01-ecc0-4298-b713-3f0756dcd15e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:06:56.576371 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:06:56.588786 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 754/1508] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:06:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:06:56.599274 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931e4e79-4bd2-4f52-a2d7-1d872b3310a2 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:06:56.600562 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-931e4e79-4bd2-4f52-a2d7-1d872b3310a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:06:56.701786 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931e4e79-4bd2-4f52-a2d7-1d872b3310a2 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:06:56.758853 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:06:56.777934 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78408e3a-f8bd-4f69-86a4-da8c59aed958 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:06:56.778543 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78408e3a-f8bd-4f69-86a4-da8c59aed958 service neutron] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:06:56.778820 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78408e3a-f8bd-4f69-86a4-da8c59aed958 service neutron] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:06:56.779091 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78408e3a-f8bd-4f69-86a4-da8c59aed958 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:06:56.780730 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-78408e3a-f8bd-4f69-86a4-da8c59aed958 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:06:56.890842 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78408e3a-f8bd-4f69-86a4-da8c59aed958 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:06:56.892201 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78408e3a-f8bd-4f69-86a4-da8c59aed958 service neutron] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:06:56.910755 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 755/1509] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:06:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:06:57.428392 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e23b01-ecc0-4298-b713-3f0756dcd15e None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:06:57.430424 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-f6e23b01-ecc0-4298-b713-3f0756dcd15e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:06:57.446590 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e23b01-ecc0-4298-b713-3f0756dcd15e None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:06:57.540699 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a4acb3-444d-4ff8-968f-07bf1ef04373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:06:57.541329 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a4acb3-444d-4ff8-968f-07bf1ef04373 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:06:57.541605 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a4acb3-444d-4ff8-968f-07bf1ef04373 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:06:57.541874 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a4acb3-444d-4ff8-968f-07bf1ef04373 admin admin] PATH_INFO: `/v3/projects/ab4494e9106c4752baebdd66ce0e2b5b` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:06:57.551564 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a4acb3-444d-4ff8-968f-07bf1ef04373 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab4494e9106c4752baebdd66ce0e2b5b)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:06:57.552619 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: WARNING py.warnings [None req-20a4acb3-444d-4ff8-968f-07bf1ef04373 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 06 22:06:57.552619 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: warnings.warn(msg) Aug 06 22:06:57.552619 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: Aug 06 22:06:57.553814 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a4acb3-444d-4ff8-968f-07bf1ef04373 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:06:57.665199 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-20a4acb3-444d-4ff8-968f-07bf1ef04373 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab4494e9106c4752baebdd66ce0e2b5b', 'request_id': 'req-20a4acb3-444d-4ff8-968f-07bf1ef04373'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:06:57.746278 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-20a4acb3-444d-4ff8-968f-07bf1ef04373 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab4494e9106c4752baebdd66ce0e2b5b', 'request_id': 'req-20a4acb3-444d-4ff8-968f-07bf1ef04373'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:06:57.752717 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-20a4acb3-444d-4ff8-968f-07bf1ef04373 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab4494e9106c4752baebdd66ce0e2b5b', 'request_id': 'req-20a4acb3-444d-4ff8-968f-07bf1ef04373'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:06:57.782535 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-20a4acb3-444d-4ff8-968f-07bf1ef04373 admin admin] The token cache is being invalidate because project ab4494e9106c4752baebdd66ce0e2b5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:06:57.782821 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-20a4acb3-444d-4ff8-968f-07bf1ef04373 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:06:57.787396 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-20a4acb3-444d-4ff8-968f-07bf1ef04373 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:06:57.787396 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 755/1510] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:06:57 2022] DELETE /identity/v3/projects/ab4494e9106c4752baebdd66ce0e2b5b => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:07:06.908610 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=62 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:07:07.780130 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:08:24.082075 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fcf32a-4050-44e8-819e-e3f32c89c534 None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:08:24.082685 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fcf32a-4050-44e8-819e-e3f32c89c534 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:08:24.082685 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fcf32a-4050-44e8-819e-e3f32c89c534 None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:08:24.084002 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 756/1511] 127.0.0.1 () {54 vars in 901 bytes} [Sat Aug 6 22:08:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:08:24.088848 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed83c877-bd13-45dd-9f48-ab144614a4b3 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:08:24.089267 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed83c877-bd13-45dd-9f48-ab144614a4b3 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:08:24.089421 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed83c877-bd13-45dd-9f48-ab144614a4b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:08:24.168661 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-ed83c877-bd13-45dd-9f48-ab144614a4b3 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:08:24.169871 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-ed83c877-bd13-45dd-9f48-ab144614a4b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:08:24.270743 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 756/1512] 127.0.0.1 () {58 vars in 998 bytes} [Sat Aug 6 22:08:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:08:24.423594 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fcf32a-4050-44e8-819e-e3f32c89c534 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:08:24.425719 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-16fcf32a-4050-44e8-819e-e3f32c89c534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:08:24.543199 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fcf32a-4050-44e8-819e-e3f32c89c534 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:08:24.548812 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e1049e-88cd-42f6-914d-5efe2c30aa7b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a1319d33e9144859c6010843be312a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:08:24.549418 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e1049e-88cd-42f6-914d-5efe2c30aa7b service nova] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:08:24.549700 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e1049e-88cd-42f6-914d-5efe2c30aa7b service nova] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:08:24.549977 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e1049e-88cd-42f6-914d-5efe2c30aa7b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:08:24.559083 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e1049e-88cd-42f6-914d-5efe2c30aa7b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:08:24.560717 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e1049e-88cd-42f6-914d-5efe2c30aa7b service nova] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:08:24.570037 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 757/1513] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:08:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:08:25.417135 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dd3553-934d-4d20-b1d2-45f88a730cc9 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:08:25.417618 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dd3553-934d-4d20-b1d2-45f88a730cc9 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:08:25.417726 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dd3553-934d-4d20-b1d2-45f88a730cc9 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:08:25.419021 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 757/1514] 127.0.0.1 () {54 vars in 901 bytes} [Sat Aug 6 22:08:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:08:25.423762 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975dba5c-8c09-4bf9-b2f5-313164185382 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:08:25.424183 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975dba5c-8c09-4bf9-b2f5-313164185382 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:08:25.424290 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975dba5c-8c09-4bf9-b2f5-313164185382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:08:25.445300 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-975dba5c-8c09-4bf9-b2f5-313164185382 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:08:25.446493 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-975dba5c-8c09-4bf9-b2f5-313164185382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:08:25.454213 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 758/1515] 127.0.0.1 () {58 vars in 998 bytes} [Sat Aug 6 22:08:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:08:25.605038 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dd3553-934d-4d20-b1d2-45f88a730cc9 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:08:25.619795 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dd3553-934d-4d20-b1d2-45f88a730cc9 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:08:25.625154 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90da0a4-0b9e-45fc-af77-1cf26413185b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a1319d33e9144859c6010843be312a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:08:25.625740 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90da0a4-0b9e-45fc-af77-1cf26413185b service nova] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:08:25.626016 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90da0a4-0b9e-45fc-af77-1cf26413185b service nova] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:08:25.626286 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90da0a4-0b9e-45fc-af77-1cf26413185b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:08:25.635328 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90da0a4-0b9e-45fc-af77-1cf26413185b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:08:25.636931 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90da0a4-0b9e-45fc-af77-1cf26413185b service nova] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:08:25.646420 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 758/1516] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:08:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:08:26.046502 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa5cc6f-9d28-43d0-b311-f63c19dbfe18 None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:08:26.046783 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa5cc6f-9d28-43d0-b311-f63c19dbfe18 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:08:26.047058 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa5cc6f-9d28-43d0-b311-f63c19dbfe18 None None] PATH_INFO: `/v3` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:08:26.048336 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 759/1517] 10.222.0.43 () {54 vars in 925 bytes} [Sat Aug 6 22:08:26 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 22:08:26.056596 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295b0128-7ef6-4e6a-acfe-c26458636478 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:08:26.056886 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295b0128-7ef6-4e6a-acfe-c26458636478 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:08:26.057164 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295b0128-7ef6-4e6a-acfe-c26458636478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:08:26.115449 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-295b0128-7ef6-4e6a-acfe-c26458636478 None None] MFA Rules not processed for user `680d62a67ae2456f8c129e26728c3297`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:08:26.116675 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-295b0128-7ef6-4e6a-acfe-c26458636478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:08:26.132834 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:08:26.188880 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 759/1518] 10.222.0.43 () {58 vars in 1015 bytes} [Sat Aug 6 22:08:26 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:08:35.439977 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=56 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:08:35.525719 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd90df1-b5b5-4e06-89d9-10516a96c0c1 None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:08:35.525925 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd90df1-b5b5-4e06-89d9-10516a96c0c1 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:08:35.526183 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd90df1-b5b5-4e06-89d9-10516a96c0c1 None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:08:35.527546 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 760/1519] 10.222.0.18 () {54 vars in 961 bytes} [Sat Aug 6 22:08:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:08:35.532712 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc52a05-468f-4cb9-a7d8-b7d336944cb0 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:08:35.533180 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc52a05-468f-4cb9-a7d8-b7d336944cb0 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:08:35.533292 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc52a05-468f-4cb9-a7d8-b7d336944cb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:08:35.552757 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-1cc52a05-468f-4cb9-a7d8-b7d336944cb0 None None] MFA Rules not processed for user `680d62a67ae2456f8c129e26728c3297`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:08:35.553940 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-1cc52a05-468f-4cb9-a7d8-b7d336944cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:08:35.562256 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 760/1520] 10.222.0.18 () {58 vars in 1058 bytes} [Sat Aug 6 22:08:35 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:08:35.570341 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd90df1-b5b5-4e06-89d9-10516a96c0c1 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:08:35.583594 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd90df1-b5b5-4e06-89d9-10516a96c0c1 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:08:35.588613 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f179a00-0fbc-4bfe-a472-ba7eaf04f40b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '680d62a67ae2456f8c129e26728c3297', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:08:35.589220 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f179a00-0fbc-4bfe-a472-ba7eaf04f40b service placement] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:08:35.589499 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f179a00-0fbc-4bfe-a472-ba7eaf04f40b service placement] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:08:35.589768 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f179a00-0fbc-4bfe-a472-ba7eaf04f40b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:08:35.597825 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f179a00-0fbc-4bfe-a472-ba7eaf04f40b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:08:35.600361 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f179a00-0fbc-4bfe-a472-ba7eaf04f40b service placement] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:08:35.610067 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 761/1521] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:08:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:08:36.872003 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64429da2-c7a6-4fcb-aa8b-ec140efbc727 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:08:36.872629 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64429da2-c7a6-4fcb-aa8b-ec140efbc727 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:08:36.872629 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64429da2-c7a6-4fcb-aa8b-ec140efbc727 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:08:36.873926 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 761/1522] 127.0.0.1 () {54 vars in 901 bytes} [Sat Aug 6 22:08:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:08:36.878839 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f7cb0c-cedf-49a1-889d-6af40d818ecf None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:08:36.879258 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f7cb0c-cedf-49a1-889d-6af40d818ecf None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:08:36.879393 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f7cb0c-cedf-49a1-889d-6af40d818ecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:08:36.899457 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-22f7cb0c-cedf-49a1-889d-6af40d818ecf None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:08:36.900636 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-22f7cb0c-cedf-49a1-889d-6af40d818ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:08:36.908341 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 762/1523] 127.0.0.1 () {58 vars in 998 bytes} [Sat Aug 6 22:08:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:08:37.062939 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64429da2-c7a6-4fcb-aa8b-ec140efbc727 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:08:37.077764 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64429da2-c7a6-4fcb-aa8b-ec140efbc727 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:08:37.083203 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b231b407-1915-48b9-90d5-bd944d4c4a15 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a1319d33e9144859c6010843be312a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:08:37.083785 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b231b407-1915-48b9-90d5-bd944d4c4a15 service nova] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:08:37.084059 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b231b407-1915-48b9-90d5-bd944d4c4a15 service nova] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:08:37.084329 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b231b407-1915-48b9-90d5-bd944d4c4a15 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:08:37.092839 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b231b407-1915-48b9-90d5-bd944d4c4a15 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:08:37.094541 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b231b407-1915-48b9-90d5-bd944d4c4a15 service nova] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:08:37.104750 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 762/1524] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:08:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:08:37.922518 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d932fe8-864f-4397-a7ca-c2b8bad7e135 None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:08:37.923136 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d932fe8-864f-4397-a7ca-c2b8bad7e135 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:08:37.923136 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d932fe8-864f-4397-a7ca-c2b8bad7e135 None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:08:37.924391 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 763/1525] 127.0.0.1 () {54 vars in 901 bytes} [Sat Aug 6 22:08:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:08:37.929187 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657a67c6-3a0d-491e-ba92-03bffc696742 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:08:37.929614 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657a67c6-3a0d-491e-ba92-03bffc696742 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:08:37.929728 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657a67c6-3a0d-491e-ba92-03bffc696742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:08:37.949005 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-657a67c6-3a0d-491e-ba92-03bffc696742 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:08:37.950196 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-657a67c6-3a0d-491e-ba92-03bffc696742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:08:37.958589 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 763/1526] 127.0.0.1 () {58 vars in 998 bytes} [Sat Aug 6 22:08:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 22:08:38.107512 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d932fe8-864f-4397-a7ca-c2b8bad7e135 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:08:38.121915 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d932fe8-864f-4397-a7ca-c2b8bad7e135 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:08:38.127394 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87559935-9b9e-433b-8352-d3040dc7a938 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a1319d33e9144859c6010843be312a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:08:38.127987 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87559935-9b9e-433b-8352-d3040dc7a938 service nova] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:08:38.128263 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87559935-9b9e-433b-8352-d3040dc7a938 service nova] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:08:38.128577 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87559935-9b9e-433b-8352-d3040dc7a938 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:08:38.137242 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87559935-9b9e-433b-8352-d3040dc7a938 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:08:38.138821 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87559935-9b9e-433b-8352-d3040dc7a938 service nova] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:08:38.148680 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 764/1527] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:08:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:08:42.196022 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9806ca9e-eb00-4f3a-9014-1f313e565814 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:08:42.196637 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9806ca9e-eb00-4f3a-9014-1f313e565814 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:08:42.196637 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9806ca9e-eb00-4f3a-9014-1f313e565814 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:08:42.197964 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 764/1528] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 6 22:08:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:08:42.202973 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3206e218-0bc1-41c2-b359-4cdd7ee1647b None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:08:42.203380 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3206e218-0bc1-41c2-b359-4cdd7ee1647b None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:08:42.203505 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3206e218-0bc1-41c2-b359-4cdd7ee1647b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:08:42.222308 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-3206e218-0bc1-41c2-b359-4cdd7ee1647b None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:08:42.223526 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-3206e218-0bc1-41c2-b359-4cdd7ee1647b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:08:42.231475 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 765/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 6 22:08:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:08:42.304461 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9806ca9e-eb00-4f3a-9014-1f313e565814 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:08:42.306180 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-9806ca9e-eb00-4f3a-9014-1f313e565814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:08:42.391704 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9806ca9e-eb00-4f3a-9014-1f313e565814 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:08:42.395446 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6eb057-6154-4fda-b8ad-b03f85f6fdec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:08:42.396163 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6eb057-6154-4fda-b8ad-b03f85f6fdec service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:08:42.396547 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6eb057-6154-4fda-b8ad-b03f85f6fdec service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:08:42.396825 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6eb057-6154-4fda-b8ad-b03f85f6fdec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:08:42.406090 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6eb057-6154-4fda-b8ad-b03f85f6fdec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:08:42.407524 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6eb057-6154-4fda-b8ad-b03f85f6fdec service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:08:42.417530 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 765/1530] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:08:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:08:44.400266 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f22caaa-bec5-4bc0-aeb2-d4301c062d19 None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:08:44.400833 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f22caaa-bec5-4bc0-aeb2-d4301c062d19 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:08:44.400940 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f22caaa-bec5-4bc0-aeb2-d4301c062d19 None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:08:44.402239 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 766/1531] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 6 22:08:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:08:44.407391 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664c6add-1d42-4b63-8b82-2ff226e2e4df None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:08:44.407779 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664c6add-1d42-4b63-8b82-2ff226e2e4df None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:08:44.407935 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664c6add-1d42-4b63-8b82-2ff226e2e4df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:08:44.427656 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-664c6add-1d42-4b63-8b82-2ff226e2e4df None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:08:44.428882 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-664c6add-1d42-4b63-8b82-2ff226e2e4df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:08:44.436618 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 766/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 6 22:08:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:08:44.472378 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f22caaa-bec5-4bc0-aeb2-d4301c062d19 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:08:44.483391 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f22caaa-bec5-4bc0-aeb2-d4301c062d19 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:08:44.486839 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e583b379-88c7-449a-a2d8-b4a7d8a5295d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:08:44.487419 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e583b379-88c7-449a-a2d8-b4a7d8a5295d service neutron] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:08:44.487694 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e583b379-88c7-449a-a2d8-b4a7d8a5295d service neutron] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:08:44.487963 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e583b379-88c7-449a-a2d8-b4a7d8a5295d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:08:44.497204 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e583b379-88c7-449a-a2d8-b4a7d8a5295d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:08:44.498613 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e583b379-88c7-449a-a2d8-b4a7d8a5295d service neutron] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:08:44.508395 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 767/1533] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:08:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:08:47.964968 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a7b18e-daba-48a6-b059-06b898182d3e None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:08:47.965541 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a7b18e-daba-48a6-b059-06b898182d3e None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:08:47.965541 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a7b18e-daba-48a6-b059-06b898182d3e None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:08:47.966859 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 767/1534] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 6 22:08:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:08:47.971958 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5735e8-3404-4fe7-b1df-4fa69390dd98 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:08:47.972387 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5735e8-3404-4fe7-b1df-4fa69390dd98 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:08:47.972551 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5735e8-3404-4fe7-b1df-4fa69390dd98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:08:47.992327 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-cc5735e8-3404-4fe7-b1df-4fa69390dd98 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:08:47.993531 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-cc5735e8-3404-4fe7-b1df-4fa69390dd98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:08:48.001141 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 768/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 6 22:08:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:08:48.309028 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a7b18e-daba-48a6-b059-06b898182d3e None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:08:48.310694 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-b3a7b18e-daba-48a6-b059-06b898182d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:08:48.375536 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:08:48.388636 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a7b18e-daba-48a6-b059-06b898182d3e None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:08:48.393962 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79618226-a27f-41a0-b801-72685dca1608 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa71c71de263478da234f3c5716a6e5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:08:48.393962 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79618226-a27f-41a0-b801-72685dca1608 service cinder] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:08:48.393962 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79618226-a27f-41a0-b801-72685dca1608 service cinder] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:08:48.393962 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79618226-a27f-41a0-b801-72685dca1608 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:08:48.402203 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79618226-a27f-41a0-b801-72685dca1608 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:08:48.403620 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79618226-a27f-41a0-b801-72685dca1608 service cinder] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:08:48.413506 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 768/1536] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:08:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 22:08:50.429696 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c066af2-a7e0-471e-b0f0-ae6467bc1b8c None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:08:50.430219 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c066af2-a7e0-471e-b0f0-ae6467bc1b8c None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:08:50.430219 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c066af2-a7e0-471e-b0f0-ae6467bc1b8c None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:08:50.431576 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 769/1537] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 6 22:08:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:08:50.436717 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7d0b97-690d-4ca1-82a3-ec005f4dd481 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:08:50.437127 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7d0b97-690d-4ca1-82a3-ec005f4dd481 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:08:50.437265 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7d0b97-690d-4ca1-82a3-ec005f4dd481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:08:50.456498 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-5e7d0b97-690d-4ca1-82a3-ec005f4dd481 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:08:50.458189 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-5e7d0b97-690d-4ca1-82a3-ec005f4dd481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:08:50.466009 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 769/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 6 22:08:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:08:50.769138 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c066af2-a7e0-471e-b0f0-ae6467bc1b8c None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:08:50.770996 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-3c066af2-a7e0-471e-b0f0-ae6467bc1b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:08:50.783633 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c066af2-a7e0-471e-b0f0-ae6467bc1b8c None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:08:50.787851 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2c176b-2b44-48f6-9f9b-3d222ba0b605 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa71c71de263478da234f3c5716a6e5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:08:50.788469 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2c176b-2b44-48f6-9f9b-3d222ba0b605 service cinder] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:08:50.788744 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2c176b-2b44-48f6-9f9b-3d222ba0b605 service cinder] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:08:50.789012 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2c176b-2b44-48f6-9f9b-3d222ba0b605 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:08:50.797789 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2c176b-2b44-48f6-9f9b-3d222ba0b605 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:08:50.799187 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2c176b-2b44-48f6-9f9b-3d222ba0b605 service cinder] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:08:50.809228 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 770/1539] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:08:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:09:00.451767 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=11 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:09:00.805011 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=33 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:10:14.489617 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be0af27-335b-4a95-9b83-39c50bb3eac2 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:14.490992 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be0af27-335b-4a95-9b83-39c50bb3eac2 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:14.491373 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be0af27-335b-4a95-9b83-39c50bb3eac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:14.517397 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-8be0af27-335b-4a95-9b83-39c50bb3eac2 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:14.518990 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-8be0af27-335b-4a95-9b83-39c50bb3eac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:14.530478 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 770/1540] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:10:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 22:10:14.539319 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e3f907-1fa6-466c-9e95-662494f49a61 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:14.540466 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e3f907-1fa6-466c-9e95-662494f49a61 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:14.541105 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e3f907-1fa6-466c-9e95-662494f49a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:14.555269 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c154e4e-b9a6-4965-bc2b-d723a7c3b34a None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:14.560844 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c154e4e-b9a6-4965-bc2b-d723a7c3b34a None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:14.561517 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c154e4e-b9a6-4965-bc2b-d723a7c3b34a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:14.579870 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-67e3f907-1fa6-466c-9e95-662494f49a61 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:14.585540 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-67e3f907-1fa6-466c-9e95-662494f49a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:14.600814 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-3c154e4e-b9a6-4965-bc2b-d723a7c3b34a None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:14.603311 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-3c154e4e-b9a6-4965-bc2b-d723a7c3b34a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:14.606042 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 771/1541] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:10:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:10:14.607586 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efde69ad-c997-448d-b048-975e211947ce None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:14.609212 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efde69ad-c997-448d-b048-975e211947ce None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:14.610698 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efde69ad-c997-448d-b048-975e211947ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:14.625604 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 771/1542] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:10:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:10:14.626736 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c154e4e-b9a6-4965-bc2b-d723a7c3b34a None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:14.634227 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-efde69ad-c997-448d-b048-975e211947ce None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:14.635874 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-efde69ad-c997-448d-b048-975e211947ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:14.645036 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c154e4e-b9a6-4965-bc2b-d723a7c3b34a None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:14.646534 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 772/1543] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:10:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:10:14.647952 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fdbd87-62b8-4b29-af06-7d3701d0f37a None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:14.648471 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fdbd87-62b8-4b29-af06-7d3701d0f37a None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:14.649035 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fdbd87-62b8-4b29-af06-7d3701d0f37a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:14.659596 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3405c0db-e953-4cdb-8bd8-614e001ff080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:14.660513 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3405c0db-e953-4cdb-8bd8-614e001ff080 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:14.660973 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3405c0db-e953-4cdb-8bd8-614e001ff080 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:14.661404 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3405c0db-e953-4cdb-8bd8-614e001ff080 admin admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:14.662288 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3405c0db-e953-4cdb-8bd8-614e001ff080 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 22:10:14.662762 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3405c0db-e953-4cdb-8bd8-614e001ff080 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:14.664583 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3405c0db-e953-4cdb-8bd8-614e001ff080 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:14.674933 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-86fdbd87-62b8-4b29-af06-7d3701d0f37a None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:14.677284 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-86fdbd87-62b8-4b29-af06-7d3701d0f37a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:14.684659 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 772/1544] 10.222.0.18 () {58 vars in 1153 bytes} [Sat Aug 6 22:10:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 22:10:14.686136 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4d458b-202f-4a64-8700-6ba9fe97046a None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:14.686598 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4d458b-202f-4a64-8700-6ba9fe97046a None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:14.687029 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4d458b-202f-4a64-8700-6ba9fe97046a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:14.687702 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 773/1545] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:10:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:10:14.689413 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aba558-2652-4e7a-a8c5-0925d0470ff7 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:14.689933 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aba558-2652-4e7a-a8c5-0925d0470ff7 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:14.690415 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aba558-2652-4e7a-a8c5-0925d0470ff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:14.713331 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-a0aba558-2652-4e7a-a8c5-0925d0470ff7 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:14.715307 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-a0aba558-2652-4e7a-a8c5-0925d0470ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:14.724260 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-3f4d458b-202f-4a64-8700-6ba9fe97046a None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:14.725785 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-3f4d458b-202f-4a64-8700-6ba9fe97046a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:14.725906 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 774/1546] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:10:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:10:14.727790 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898d6f25-6fcd-44fa-b76a-82c514c7e80b None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:14.728292 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898d6f25-6fcd-44fa-b76a-82c514c7e80b None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:14.728808 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898d6f25-6fcd-44fa-b76a-82c514c7e80b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:14.743775 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 773/1547] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:10:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:10:14.744648 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4d458b-202f-4a64-8700-6ba9fe97046a None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:14.753000 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-898d6f25-6fcd-44fa-b76a-82c514c7e80b None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:14.754199 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-898d6f25-6fcd-44fa-b76a-82c514c7e80b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:14.758717 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4d458b-202f-4a64-8700-6ba9fe97046a None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:14.762183 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 775/1548] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:10:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:10:14.763345 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa30ac88-ceae-4afd-b73c-ae09a648e73e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:14.763475 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e800a3-76d4-42cc-afb8-ef49c0271049 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:14.763735 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e800a3-76d4-42cc-afb8-ef49c0271049 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:14.763951 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa30ac88-ceae-4afd-b73c-ae09a648e73e admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:14.764075 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e800a3-76d4-42cc-afb8-ef49c0271049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:14.764239 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa30ac88-ceae-4afd-b73c-ae09a648e73e admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:14.764537 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa30ac88-ceae-4afd-b73c-ae09a648e73e admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:14.765167 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa30ac88-ceae-4afd-b73c-ae09a648e73e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:14.767056 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: WARNING py.warnings [None req-aa30ac88-ceae-4afd-b73c-ae09a648e73e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 06 22:10:14.767056 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: warnings.warn(msg) Aug 06 22:10:14.767056 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: Aug 06 22:10:14.768270 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa30ac88-ceae-4afd-b73c-ae09a648e73e admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:14.784046 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-b9e800a3-76d4-42cc-afb8-ef49c0271049 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:14.785252 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-b9e800a3-76d4-42cc-afb8-ef49c0271049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:14.793132 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 776/1549] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:10:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:10:14.794433 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac14eef7-4d00-4d67-8888-40779b703a58 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:14.794714 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac14eef7-4d00-4d67-8888-40779b703a58 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:14.794985 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac14eef7-4d00-4d67-8888-40779b703a58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:14.815589 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-ac14eef7-4d00-4d67-8888-40779b703a58 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:14.816891 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-ac14eef7-4d00-4d67-8888-40779b703a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:14.824882 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 777/1550] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:10:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:10:14.825997 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac14eef7-4d00-4d67-8888-40779b703a58 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:14.835306 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac14eef7-4d00-4d67-8888-40779b703a58 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:14.839818 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56ee5ba-69d2-43b8-b9a0-dc8a3a747fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:14.840382 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ee5ba-69d2-43b8-b9a0-dc8a3a747fc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:14.840742 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ee5ba-69d2-43b8-b9a0-dc8a3a747fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:14.841009 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ee5ba-69d2-43b8-b9a0-dc8a3a747fc0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:14.841636 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56ee5ba-69d2-43b8-b9a0-dc8a3a747fc0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 22:10:14.841936 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56ee5ba-69d2-43b8-b9a0-dc8a3a747fc0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:14.842933 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: WARNING py.warnings [None req-c56ee5ba-69d2-43b8-b9a0-dc8a3a747fc0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 06 22:10:14.842933 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: warnings.warn(msg) Aug 06 22:10:14.842933 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: Aug 06 22:10:14.843930 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56ee5ba-69d2-43b8-b9a0-dc8a3a747fc0 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:14.856444 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 778/1551] 10.222.0.18 () {58 vars in 1153 bytes} [Sat Aug 6 22:10:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 22:10:14.856817 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e5253-fe8e-4a1b-9ebc-4038880880c2 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:14.857107 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e5253-fe8e-4a1b-9ebc-4038880880c2 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:14.857383 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e5253-fe8e-4a1b-9ebc-4038880880c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:14.877410 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-b43e5253-fe8e-4a1b-9ebc-4038880880c2 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:14.879094 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-b43e5253-fe8e-4a1b-9ebc-4038880880c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:14.885028 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 774/1552] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:10:14 2022] POST /identity/v3/projects => generated 385 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:10:14.886447 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d6c71-ded0-43fa-8f35-2c915ed9ee28 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:14.886883 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d6c71-ded0-43fa-8f35-2c915ed9ee28 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:14.887161 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d6c71-ded0-43fa-8f35-2c915ed9ee28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:14.890344 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 779/1553] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:10:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:10:14.905556 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd18723a-25db-4641-8a76-021aa0894e59 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:14.905556 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd18723a-25db-4641-8a76-021aa0894e59 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:14.905556 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd18723a-25db-4641-8a76-021aa0894e59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:14.911555 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-2b8d6c71-ded0-43fa-8f35-2c915ed9ee28 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:14.912666 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-cd18723a-25db-4641-8a76-021aa0894e59 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:14.912800 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-2b8d6c71-ded0-43fa-8f35-2c915ed9ee28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:14.913869 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-cd18723a-25db-4641-8a76-021aa0894e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:14.972953 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 775/1554] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:10:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:10:14.973854 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 780/1555] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:10:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:10:14.974258 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8d6c71-ded0-43fa-8f35-2c915ed9ee28 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:14.977750 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcba909-435d-4662-9ef9-d0e2aad285aa None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:14.978294 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcba909-435d-4662-9ef9-d0e2aad285aa None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:14.978605 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcba909-435d-4662-9ef9-d0e2aad285aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:14.986092 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8d6c71-ded0-43fa-8f35-2c915ed9ee28 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:14.990593 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a530b506-8241-46cd-98b5-fbc489f4f1ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:14.991222 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a530b506-8241-46cd-98b5-fbc489f4f1ab admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:14.991483 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a530b506-8241-46cd-98b5-fbc489f4f1ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:14.991750 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a530b506-8241-46cd-98b5-fbc489f4f1ab admin admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:14.992415 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a530b506-8241-46cd-98b5-fbc489f4f1ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 22:10:14.992826 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a530b506-8241-46cd-98b5-fbc489f4f1ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:14.994409 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a530b506-8241-46cd-98b5-fbc489f4f1ab admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:14.999080 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-cdcba909-435d-4662-9ef9-d0e2aad285aa None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:15.000568 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-cdcba909-435d-4662-9ef9-d0e2aad285aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:15.006807 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 776/1556] 10.222.0.18 () {58 vars in 1153 bytes} [Sat Aug 6 22:10:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 22:10:15.007872 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a530b506-8241-46cd-98b5-fbc489f4f1ab admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:15.014477 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 781/1557] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:10:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:10:15.014477 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcba909-435d-4662-9ef9-d0e2aad285aa None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:15.019798 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a530b506-8241-46cd-98b5-fbc489f4f1ab admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:15.023884 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcba909-435d-4662-9ef9-d0e2aad285aa None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:15.024327 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc325b8-3b65-4345-9373-588a8401d4d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:15.024945 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc325b8-3b65-4345-9373-588a8401d4d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:15.025223 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc325b8-3b65-4345-9373-588a8401d4d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:15.025496 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc325b8-3b65-4345-9373-588a8401d4d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:15.026130 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc325b8-3b65-4345-9373-588a8401d4d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 22:10:15.026436 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc325b8-3b65-4345-9373-588a8401d4d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:15.028159 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc325b8-3b65-4345-9373-588a8401d4d3 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:15.028575 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237c5aba-18f8-4cd4-bd8f-680c277e03a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:15.029188 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237c5aba-18f8-4cd4-bd8f-680c277e03a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:15.029472 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237c5aba-18f8-4cd4-bd8f-680c277e03a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:15.029736 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237c5aba-18f8-4cd4-bd8f-680c277e03a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:15.030354 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237c5aba-18f8-4cd4-bd8f-680c277e03a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:15.032974 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237c5aba-18f8-4cd4-bd8f-680c277e03a2 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:15.041341 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 777/1558] 10.222.0.18 () {58 vars in 1153 bytes} [Sat Aug 6 22:10:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 22:10:15.041341 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc325b8-3b65-4345-9373-588a8401d4d3 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:15.054921 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc325b8-3b65-4345-9373-588a8401d4d3 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:15.059526 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5399e2b5-4e22-4599-a1c0-2723b4e6dd32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:15.062594 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5399e2b5-4e22-4599-a1c0-2723b4e6dd32 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:15.062594 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5399e2b5-4e22-4599-a1c0-2723b4e6dd32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:15.062594 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5399e2b5-4e22-4599-a1c0-2723b4e6dd32 admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:15.062594 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5399e2b5-4e22-4599-a1c0-2723b4e6dd32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:15.063252 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5399e2b5-4e22-4599-a1c0-2723b4e6dd32 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:15.154646 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 782/1559] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:10:15 2022] POST /identity/v3/projects => generated 373 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:10:15.154646 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9c5dda-bfbb-4f1d-a5a9-4df5d628c08a None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:15.154646 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9c5dda-bfbb-4f1d-a5a9-4df5d628c08a None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:15.154646 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9c5dda-bfbb-4f1d-a5a9-4df5d628c08a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:15.173087 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-cf9c5dda-bfbb-4f1d-a5a9-4df5d628c08a None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:15.177419 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-cf9c5dda-bfbb-4f1d-a5a9-4df5d628c08a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:15.179879 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 778/1560] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:10:15 2022] POST /identity/v3/users => generated 293 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:10:15.180567 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5399e2b5-4e22-4599-a1c0-2723b4e6dd32 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:15.190418 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5399e2b5-4e22-4599-a1c0-2723b4e6dd32 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:15.236742 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 783/1561] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:10:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:10:15.238048 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcea588-9efe-4f93-8e7a-65da4f1fa5a0 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:15.238417 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcea588-9efe-4f93-8e7a-65da4f1fa5a0 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:15.238606 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcea588-9efe-4f93-8e7a-65da4f1fa5a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:15.239640 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:10:15.249813 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669bf74e-fe92-47a8-b759-b2f2f6d1e7b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:15.250637 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669bf74e-fe92-47a8-b759-b2f2f6d1e7b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:15.250914 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669bf74e-fe92-47a8-b759-b2f2f6d1e7b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:15.251183 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669bf74e-fe92-47a8-b759-b2f2f6d1e7b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:15.251851 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669bf74e-fe92-47a8-b759-b2f2f6d1e7b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 22:10:15.252149 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669bf74e-fe92-47a8-b759-b2f2f6d1e7b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:15.253011 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:10:15.253962 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669bf74e-fe92-47a8-b759-b2f2f6d1e7b3 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:15.261276 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-0fcea588-9efe-4f93-8e7a-65da4f1fa5a0 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:15.262487 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-0fcea588-9efe-4f93-8e7a-65da4f1fa5a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:15.273800 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 779/1562] 10.222.0.18 () {58 vars in 1153 bytes} [Sat Aug 6 22:10:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 22:10:15.273800 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669bf74e-fe92-47a8-b759-b2f2f6d1e7b3 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:15.277567 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669bf74e-fe92-47a8-b759-b2f2f6d1e7b3 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:15.283080 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 784/1563] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:10:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:10:15.283080 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcea588-9efe-4f93-8e7a-65da4f1fa5a0 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:15.283300 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775c9788-b32e-4698-b1c1-d3e897821920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:15.283596 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775c9788-b32e-4698-b1c1-d3e897821920 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:15.283872 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775c9788-b32e-4698-b1c1-d3e897821920 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:15.284137 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775c9788-b32e-4698-b1c1-d3e897821920 admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:15.284810 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775c9788-b32e-4698-b1c1-d3e897821920 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:15.286697 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775c9788-b32e-4698-b1c1-d3e897821920 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:15.293059 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcea588-9efe-4f93-8e7a-65da4f1fa5a0 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:15.297978 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cdf398-b83d-4223-b29c-44e12e2c10bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:15.298618 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cdf398-b83d-4223-b29c-44e12e2c10bf admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:15.298896 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cdf398-b83d-4223-b29c-44e12e2c10bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:15.299159 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cdf398-b83d-4223-b29c-44e12e2c10bf admin admin] PATH_INFO: `/v3/domains` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:15.299799 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cdf398-b83d-4223-b29c-44e12e2c10bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 22:10:15.300095 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cdf398-b83d-4223-b29c-44e12e2c10bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:15.301700 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cdf398-b83d-4223-b29c-44e12e2c10bf admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:15.313157 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 785/1564] 10.222.0.18 () {58 vars in 1153 bytes} [Sat Aug 6 22:10:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 22:10:15.314055 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cdf398-b83d-4223-b29c-44e12e2c10bf admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:15.323801 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cdf398-b83d-4223-b29c-44e12e2c10bf admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:15.329029 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28a6f81-8ed1-4b0b-8b23-f6f504afa9ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:15.329608 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28a6f81-8ed1-4b0b-8b23-f6f504afa9ea admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:15.329883 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28a6f81-8ed1-4b0b-8b23-f6f504afa9ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:15.330156 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28a6f81-8ed1-4b0b-8b23-f6f504afa9ea admin admin] PATH_INFO: `/v3/projects` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:15.330761 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28a6f81-8ed1-4b0b-8b23-f6f504afa9ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:15.332642 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28a6f81-8ed1-4b0b-8b23-f6f504afa9ea admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:15.393269 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 780/1565] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:10:15 2022] POST /identity/v3/projects => generated 367 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:10:15.394079 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775c9788-b32e-4698-b1c1-d3e897821920 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:15.423982 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 786/1566] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:10:15 2022] POST /identity/v3/projects => generated 373 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:10:15.423982 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28a6f81-8ed1-4b0b-8b23-f6f504afa9ea admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:15.462217 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775c9788-b32e-4698-b1c1-d3e897821920 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:15.485017 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28a6f81-8ed1-4b0b-8b23-f6f504afa9ea admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:15.490154 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430715e3-9f08-44dd-8f8f-c7d98ad24cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:15.490787 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430715e3-9f08-44dd-8f8f-c7d98ad24cd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:15.491091 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430715e3-9f08-44dd-8f8f-c7d98ad24cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:15.491364 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430715e3-9f08-44dd-8f8f-c7d98ad24cd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:15.492007 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430715e3-9f08-44dd-8f8f-c7d98ad24cd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:15.493637 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430715e3-9f08-44dd-8f8f-c7d98ad24cd9 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:15.503706 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 787/1567] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:10:15 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:10:15.504570 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430715e3-9f08-44dd-8f8f-c7d98ad24cd9 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:15.514251 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430715e3-9f08-44dd-8f8f-c7d98ad24cd9 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:15.520044 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71369b28-8f42-44c8-845f-90eb4485a76c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:15.520661 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71369b28-8f42-44c8-845f-90eb4485a76c admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:15.520943 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71369b28-8f42-44c8-845f-90eb4485a76c admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:15.521214 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71369b28-8f42-44c8-845f-90eb4485a76c admin admin] PATH_INFO: `/v3/domains` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:15.521896 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71369b28-8f42-44c8-845f-90eb4485a76c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 22:10:15.522204 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71369b28-8f42-44c8-845f-90eb4485a76c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:15.522303 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e242af-1322-4511-910d-c2f750e00776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:15.522844 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e242af-1322-4511-910d-c2f750e00776 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:15.523125 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e242af-1322-4511-910d-c2f750e00776 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:15.523397 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e242af-1322-4511-910d-c2f750e00776 admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:15.523787 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71369b28-8f42-44c8-845f-90eb4485a76c admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:15.524014 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e242af-1322-4511-910d-c2f750e00776 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:15.525969 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e242af-1322-4511-910d-c2f750e00776 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:15.536894 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 788/1568] 10.222.0.18 () {58 vars in 1153 bytes} [Sat Aug 6 22:10:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 22:10:15.537778 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71369b28-8f42-44c8-845f-90eb4485a76c admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:15.547310 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71369b28-8f42-44c8-845f-90eb4485a76c admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:15.552257 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ae9c2e-4157-40e7-8965-4c5b28ab7244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:15.552878 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ae9c2e-4157-40e7-8965-4c5b28ab7244 admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:15.553156 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ae9c2e-4157-40e7-8965-4c5b28ab7244 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:15.553431 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ae9c2e-4157-40e7-8965-4c5b28ab7244 admin admin] PATH_INFO: `/v3/projects` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:15.554049 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ae9c2e-4157-40e7-8965-4c5b28ab7244 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:15.556071 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ae9c2e-4157-40e7-8965-4c5b28ab7244 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:15.630995 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 781/1569] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:10:15 2022] POST /identity/v3/users => generated 287 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:10:15.631955 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e242af-1322-4511-910d-c2f750e00776 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:15.641909 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e242af-1322-4511-910d-c2f750e00776 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:15.646425 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea23a246-8b7f-42f6-b3d5-ea2426dbec24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:15.646994 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea23a246-8b7f-42f6-b3d5-ea2426dbec24 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:15.647269 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea23a246-8b7f-42f6-b3d5-ea2426dbec24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:15.647536 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea23a246-8b7f-42f6-b3d5-ea2426dbec24 admin admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:15.648156 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea23a246-8b7f-42f6-b3d5-ea2426dbec24 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 22:10:15.648478 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea23a246-8b7f-42f6-b3d5-ea2426dbec24 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:15.650061 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea23a246-8b7f-42f6-b3d5-ea2426dbec24 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:15.662540 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 782/1570] 10.222.0.18 () {58 vars in 1153 bytes} [Sat Aug 6 22:10:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 22:10:15.663309 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea23a246-8b7f-42f6-b3d5-ea2426dbec24 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:15.669442 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 789/1571] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:10:15 2022] POST /identity/v3/projects => generated 389 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:10:15.669442 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ae9c2e-4157-40e7-8965-4c5b28ab7244 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:15.677439 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea23a246-8b7f-42f6-b3d5-ea2426dbec24 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:15.722058 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:10:15.733501 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d8ff1b-1323-457b-a96c-f70c38e1ee40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:15.735198 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d8ff1b-1323-457b-a96c-f70c38e1ee40 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:15.735198 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d8ff1b-1323-457b-a96c-f70c38e1ee40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:15.735198 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d8ff1b-1323-457b-a96c-f70c38e1ee40 admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:15.735624 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ae9c2e-4157-40e7-8965-4c5b28ab7244 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:15.735723 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d8ff1b-1323-457b-a96c-f70c38e1ee40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:15.737220 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d8ff1b-1323-457b-a96c-f70c38e1ee40 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:15.739851 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e17ba9-8192-41e6-b34c-46a9aad1e84b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:15.740478 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e17ba9-8192-41e6-b34c-46a9aad1e84b admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:15.740765 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e17ba9-8192-41e6-b34c-46a9aad1e84b admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:15.741037 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e17ba9-8192-41e6-b34c-46a9aad1e84b admin admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:15.741659 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e17ba9-8192-41e6-b34c-46a9aad1e84b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:15.742666 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: WARNING py.warnings [None req-46e17ba9-8192-41e6-b34c-46a9aad1e84b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 06 22:10:15.742666 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: warnings.warn(msg) Aug 06 22:10:15.742666 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: Aug 06 22:10:15.743873 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e17ba9-8192-41e6-b34c-46a9aad1e84b admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:15.829346 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 790/1572] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:10:15 2022] POST /identity/v3/users => generated 284 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:10:15.829346 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e17ba9-8192-41e6-b34c-46a9aad1e84b admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:15.834830 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 783/1573] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:10:15 2022] POST /identity/v3/projects => generated 355 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:10:15.834830 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d8ff1b-1323-457b-a96c-f70c38e1ee40 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:15.839486 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e17ba9-8192-41e6-b34c-46a9aad1e84b admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:15.851431 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:10:15.897032 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbffc3a9-b51b-4fbc-bdb0-32506a7eae5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:15.897640 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbffc3a9-b51b-4fbc-bdb0-32506a7eae5c admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:15.897920 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbffc3a9-b51b-4fbc-bdb0-32506a7eae5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:15.898200 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbffc3a9-b51b-4fbc-bdb0-32506a7eae5c admin admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:15.898843 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbffc3a9-b51b-4fbc-bdb0-32506a7eae5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:15.898962 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d8ff1b-1323-457b-a96c-f70c38e1ee40 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:15.901134 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbffc3a9-b51b-4fbc-bdb0-32506a7eae5c admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:15.903726 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a39035-4ab6-4778-811e-50f6ab4c1c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:15.904319 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a39035-4ab6-4778-811e-50f6ab4c1c30 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:15.904622 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a39035-4ab6-4778-811e-50f6ab4c1c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:15.904900 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a39035-4ab6-4778-811e-50f6ab4c1c30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:15.905539 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a39035-4ab6-4778-811e-50f6ab4c1c30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:15.907651 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a39035-4ab6-4778-811e-50f6ab4c1c30 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:15.917573 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 784/1574] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:10:15 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:10:15.918345 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a39035-4ab6-4778-811e-50f6ab4c1c30 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:15.927793 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a39035-4ab6-4778-811e-50f6ab4c1c30 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:15.932113 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe642dc-1872-4bf5-adc9-e0f27e34da05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:15.932735 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe642dc-1872-4bf5-adc9-e0f27e34da05 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:15.933004 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe642dc-1872-4bf5-adc9-e0f27e34da05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:15.933274 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe642dc-1872-4bf5-adc9-e0f27e34da05 admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:15.933872 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe642dc-1872-4bf5-adc9-e0f27e34da05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:15.935728 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe642dc-1872-4bf5-adc9-e0f27e34da05 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:16.112040 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 791/1575] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:10:15 2022] POST /identity/v3/users => generated 287 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:10:16.112905 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbffc3a9-b51b-4fbc-bdb0-32506a7eae5c admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:16.122266 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbffc3a9-b51b-4fbc-bdb0-32506a7eae5c admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:16.127403 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929cc79b-0ebe-4a98-bee6-331c40abaa2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:16.127976 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929cc79b-0ebe-4a98-bee6-331c40abaa2b admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:16.128255 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929cc79b-0ebe-4a98-bee6-331c40abaa2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:16.128567 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929cc79b-0ebe-4a98-bee6-331c40abaa2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:16.129193 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929cc79b-0ebe-4a98-bee6-331c40abaa2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:16.130778 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929cc79b-0ebe-4a98-bee6-331c40abaa2b admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:16.135860 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 785/1576] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:10:15 2022] POST /identity/v3/projects => generated 377 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:10:16.136706 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe642dc-1872-4bf5-adc9-e0f27e34da05 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:16.146832 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 792/1577] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:10:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:10:16.146832 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929cc79b-0ebe-4a98-bee6-331c40abaa2b admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:16.154145 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929cc79b-0ebe-4a98-bee6-331c40abaa2b admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:16.202482 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe642dc-1872-4bf5-adc9-e0f27e34da05 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:16.207211 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92ce645-3323-4c28-8131-5b7c899eb73a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:16.207811 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92ce645-3323-4c28-8131-5b7c899eb73a admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:16.208087 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92ce645-3323-4c28-8131-5b7c899eb73a admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:16.208355 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92ce645-3323-4c28-8131-5b7c899eb73a admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:16.208996 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92ce645-3323-4c28-8131-5b7c899eb73a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:16.209930 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b701c69-9fd0-4d81-bf6f-62ca1b130281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:16.210621 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b701c69-9fd0-4d81-bf6f-62ca1b130281 admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:16.210900 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92ce645-3323-4c28-8131-5b7c899eb73a admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:16.211002 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b701c69-9fd0-4d81-bf6f-62ca1b130281 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:16.211174 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b701c69-9fd0-4d81-bf6f-62ca1b130281 admin admin] PATH_INFO: `/v3/projects` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:16.211799 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b701c69-9fd0-4d81-bf6f-62ca1b130281 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:16.213729 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b701c69-9fd0-4d81-bf6f-62ca1b130281 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:16.318177 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 786/1578] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:10:16 2022] POST /identity/v3/users => generated 295 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:10:16.319012 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92ce645-3323-4c28-8131-5b7c899eb73a admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:16.328427 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92ce645-3323-4c28-8131-5b7c899eb73a admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:16.332940 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72863e5-709f-40b2-9a1f-b6aaabd30332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:16.333509 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72863e5-709f-40b2-9a1f-b6aaabd30332 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:16.333782 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72863e5-709f-40b2-9a1f-b6aaabd30332 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:16.334049 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72863e5-709f-40b2-9a1f-b6aaabd30332 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:16.334664 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72863e5-709f-40b2-9a1f-b6aaabd30332 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:16.336225 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72863e5-709f-40b2-9a1f-b6aaabd30332 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:16.346121 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 787/1579] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:10:16 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:10:16.346121 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72863e5-709f-40b2-9a1f-b6aaabd30332 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:16.357243 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72863e5-709f-40b2-9a1f-b6aaabd30332 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:16.357851 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 793/1580] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:10:16 2022] POST /identity/v3/projects => generated 365 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:10:16.362944 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b701c69-9fd0-4d81-bf6f-62ca1b130281 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:16.385143 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:10:16.416531 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ae55de-4acb-4aea-b2f2-b40cb3bec44c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:16.417190 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ae55de-4acb-4aea-b2f2-b40cb3bec44c admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:16.417471 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ae55de-4acb-4aea-b2f2-b40cb3bec44c admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:16.417750 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ae55de-4acb-4aea-b2f2-b40cb3bec44c admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:16.418371 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ae55de-4acb-4aea-b2f2-b40cb3bec44c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:16.420294 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ae55de-4acb-4aea-b2f2-b40cb3bec44c admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:16.426232 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b701c69-9fd0-4d81-bf6f-62ca1b130281 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:16.430944 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2c2a2f-ed44-4f4b-aa48-1f484a26f081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:16.431595 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2c2a2f-ed44-4f4b-aa48-1f484a26f081 admin admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:16.431871 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2c2a2f-ed44-4f4b-aa48-1f484a26f081 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:16.432142 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2c2a2f-ed44-4f4b-aa48-1f484a26f081 admin admin] PATH_INFO: `/v3/projects/f4867885fc0d4ca29d2d674465398466/users/0d6e35038fed4b44b986902a5ef55ce8/roles/56e4a96094e9434cbbafb22579c034f5` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:16.447117 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2c2a2f-ed44-4f4b-aa48-1f484a26f081 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4867885fc0d4ca29d2d674465398466, user_id=0d6e35038fed4b44b986902a5ef55ce8, role_id=56e4a96094e9434cbbafb22579c034f5)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:16.449807 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2c2a2f-ed44-4f4b-aa48-1f484a26f081 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:16.499777 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 788/1581] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:10:16 2022] POST /identity/v3/users => generated 278 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:10:16.500863 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ae55de-4acb-4aea-b2f2-b40cb3bec44c admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:16.510569 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ae55de-4acb-4aea-b2f2-b40cb3bec44c admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:16.515181 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe9e2d3-d6ab-4fba-8f36-8c570d29fd33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:16.515839 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe9e2d3-d6ab-4fba-8f36-8c570d29fd33 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:16.516114 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe9e2d3-d6ab-4fba-8f36-8c570d29fd33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:16.516383 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe9e2d3-d6ab-4fba-8f36-8c570d29fd33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:16.517026 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe9e2d3-d6ab-4fba-8f36-8c570d29fd33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:16.518599 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe9e2d3-d6ab-4fba-8f36-8c570d29fd33 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:16.528426 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 794/1582] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:10:16 2022] PUT /identity/v3/projects/f4867885fc0d4ca29d2d674465398466/users/0d6e35038fed4b44b986902a5ef55ce8/roles/56e4a96094e9434cbbafb22579c034f5 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:10:16.536725 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 789/1583] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:10:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:10:16.537210 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2c2a2f-ed44-4f4b-aa48-1f484a26f081 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:16.537340 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe9e2d3-d6ab-4fba-8f36-8c570d29fd33 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:16.538011 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:10:16.540527 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe9e2d3-d6ab-4fba-8f36-8c570d29fd33 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:16.596587 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ad81d7-93d0-436b-8b18-76ada0ffef8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:16.597213 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ad81d7-93d0-436b-8b18-76ada0ffef8a admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:16.597489 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ad81d7-93d0-436b-8b18-76ada0ffef8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:16.597760 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ad81d7-93d0-436b-8b18-76ada0ffef8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:16.598383 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2c2a2f-ed44-4f4b-aa48-1f484a26f081 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:16.598827 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ad81d7-93d0-436b-8b18-76ada0ffef8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:16.600078 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ad81d7-93d0-436b-8b18-76ada0ffef8a admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:16.603542 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69e72d5-5189-488f-9ae9-165434afc2b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:16.604149 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69e72d5-5189-488f-9ae9-165434afc2b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:16.604602 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69e72d5-5189-488f-9ae9-165434afc2b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:16.604909 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69e72d5-5189-488f-9ae9-165434afc2b2 admin admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:16.605540 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69e72d5-5189-488f-9ae9-165434afc2b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:16.607624 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69e72d5-5189-488f-9ae9-165434afc2b2 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:16.610872 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 790/1584] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:10:16 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:10:16.611778 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ad81d7-93d0-436b-8b18-76ada0ffef8a admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:16.621448 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ad81d7-93d0-436b-8b18-76ada0ffef8a admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:16.626224 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb62c62c-6d49-4a2c-b1c0-ae93d3296dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:16.626803 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb62c62c-6d49-4a2c-b1c0-ae93d3296dbd admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:16.627078 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb62c62c-6d49-4a2c-b1c0-ae93d3296dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:16.627375 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb62c62c-6d49-4a2c-b1c0-ae93d3296dbd admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:16.627998 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb62c62c-6d49-4a2c-b1c0-ae93d3296dbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:16.629703 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb62c62c-6d49-4a2c-b1c0-ae93d3296dbd admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:16.639261 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 791/1585] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:10:16 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:10:16.640897 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb62c62c-6d49-4a2c-b1c0-ae93d3296dbd admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:16.650181 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb62c62c-6d49-4a2c-b1c0-ae93d3296dbd admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:16.654762 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b393b1-fbec-4765-a59b-b221db437d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:16.655329 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b393b1-fbec-4765-a59b-b221db437d42 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:16.655611 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b393b1-fbec-4765-a59b-b221db437d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:16.655887 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b393b1-fbec-4765-a59b-b221db437d42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:16.657049 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b393b1-fbec-4765-a59b-b221db437d42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:16.658686 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b393b1-fbec-4765-a59b-b221db437d42 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:16.669115 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 792/1586] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:10:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:10:16.669935 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b393b1-fbec-4765-a59b-b221db437d42 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:16.679826 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b393b1-fbec-4765-a59b-b221db437d42 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:16.684630 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c8ca97-cff5-4ed7-8b04-ba03caad8933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:16.693037 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c8ca97-cff5-4ed7-8b04-ba03caad8933 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:16.693037 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c8ca97-cff5-4ed7-8b04-ba03caad8933 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:16.693037 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c8ca97-cff5-4ed7-8b04-ba03caad8933 admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:16.693037 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c8ca97-cff5-4ed7-8b04-ba03caad8933 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:16.693037 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c8ca97-cff5-4ed7-8b04-ba03caad8933 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:16.693490 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 795/1587] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:10:16 2022] POST /identity/v3/users => generated 289 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:10:16.693490 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69e72d5-5189-488f-9ae9-165434afc2b2 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:16.699689 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69e72d5-5189-488f-9ae9-165434afc2b2 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:16.704473 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4cf159-910c-4d32-a9b6-2b610a7c33a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:16.705072 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4cf159-910c-4d32-a9b6-2b610a7c33a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:16.705353 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4cf159-910c-4d32-a9b6-2b610a7c33a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:16.705629 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4cf159-910c-4d32-a9b6-2b610a7c33a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:16.706323 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4cf159-910c-4d32-a9b6-2b610a7c33a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:16.707895 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4cf159-910c-4d32-a9b6-2b610a7c33a8 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:16.718089 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 796/1588] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:10:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:10:16.720124 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc4004f-249d-4a51-af8e-9dab4b9b0203 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:16.720431 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc4004f-249d-4a51-af8e-9dab4b9b0203 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:16.720706 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc4004f-249d-4a51-af8e-9dab4b9b0203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:16.739492 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-1dc4004f-249d-4a51-af8e-9dab4b9b0203 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:16.740867 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-1dc4004f-249d-4a51-af8e-9dab4b9b0203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:16.748329 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 797/1589] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:10:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:10:16.749119 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc4004f-249d-4a51-af8e-9dab4b9b0203 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:16.758929 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc4004f-249d-4a51-af8e-9dab4b9b0203 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:16.764421 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a571a1a5-e8e7-4d3f-825d-8b399bf183a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:16.765041 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a571a1a5-e8e7-4d3f-825d-8b399bf183a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:16.765316 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a571a1a5-e8e7-4d3f-825d-8b399bf183a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:16.765688 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a571a1a5-e8e7-4d3f-825d-8b399bf183a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:16.766316 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a571a1a5-e8e7-4d3f-825d-8b399bf183a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:16.767898 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a571a1a5-e8e7-4d3f-825d-8b399bf183a1 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:16.778417 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 798/1590] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:10:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:10:16.779322 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a571a1a5-e8e7-4d3f-825d-8b399bf183a1 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:16.783613 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 793/1591] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:10:16 2022] POST /identity/v3/users => generated 283 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:10:16.783613 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c8ca97-cff5-4ed7-8b04-ba03caad8933 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:16.790590 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a571a1a5-e8e7-4d3f-825d-8b399bf183a1 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:16.793061 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c8ca97-cff5-4ed7-8b04-ba03caad8933 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:16.795146 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dae899-bc60-4603-8623-0d3107a0eb2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:16.795776 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dae899-bc60-4603-8623-0d3107a0eb2a admin admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:16.796048 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dae899-bc60-4603-8623-0d3107a0eb2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:16.796313 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dae899-bc60-4603-8623-0d3107a0eb2a admin admin] PATH_INFO: `/v3/projects/2d9aaf79afbe4f87a96eae7bb7d6b8e3/users/2625522503744c9993b08b326a6c5e7f/roles/56e4a96094e9434cbbafb22579c034f5` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:16.798458 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeba3f4-577f-4978-8de4-dfca887bb0e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:16.799049 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeba3f4-577f-4978-8de4-dfca887bb0e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:16.799335 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeba3f4-577f-4978-8de4-dfca887bb0e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:16.799612 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeba3f4-577f-4978-8de4-dfca887bb0e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:16.800243 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeba3f4-577f-4978-8de4-dfca887bb0e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:16.801938 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeba3f4-577f-4978-8de4-dfca887bb0e3 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:16.812137 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dae899-bc60-4603-8623-0d3107a0eb2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d9aaf79afbe4f87a96eae7bb7d6b8e3, user_id=2625522503744c9993b08b326a6c5e7f, role_id=56e4a96094e9434cbbafb22579c034f5)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:16.812498 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 794/1592] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:10:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:10:16.813456 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeba3f4-577f-4978-8de4-dfca887bb0e3 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:16.815376 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dae899-bc60-4603-8623-0d3107a0eb2a admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:16.823157 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeba3f4-577f-4978-8de4-dfca887bb0e3 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:16.827785 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df253821-cf74-433c-82bb-de41d21cb996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:16.829293 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df253821-cf74-433c-82bb-de41d21cb996 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:16.829588 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df253821-cf74-433c-82bb-de41d21cb996 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:16.829865 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df253821-cf74-433c-82bb-de41d21cb996 admin admin] PATH_INFO: `/v3/projects/eac4e8cd9e2c4d0c8a98555d231f049b/users/43c7063420214750b53126a323c85e12/roles/56e4a96094e9434cbbafb22579c034f5` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:16.844433 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df253821-cf74-433c-82bb-de41d21cb996 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eac4e8cd9e2c4d0c8a98555d231f049b, user_id=43c7063420214750b53126a323c85e12, role_id=56e4a96094e9434cbbafb22579c034f5)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:16.845458 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: WARNING py.warnings [None req-df253821-cf74-433c-82bb-de41d21cb996 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 06 22:10:16.845458 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: warnings.warn(msg) Aug 06 22:10:16.845458 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: Aug 06 22:10:16.847424 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df253821-cf74-433c-82bb-de41d21cb996 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:16.883495 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 799/1593] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:10:16 2022] PUT /identity/v3/projects/2d9aaf79afbe4f87a96eae7bb7d6b8e3/users/2625522503744c9993b08b326a6c5e7f/roles/56e4a96094e9434cbbafb22579c034f5 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:10:16.884462 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dae899-bc60-4603-8623-0d3107a0eb2a admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:16.906236 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 795/1594] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:10:16 2022] PUT /identity/v3/projects/eac4e8cd9e2c4d0c8a98555d231f049b/users/43c7063420214750b53126a323c85e12/roles/56e4a96094e9434cbbafb22579c034f5 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:10:16.907271 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df253821-cf74-433c-82bb-de41d21cb996 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:16.938948 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:10:16.947675 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dae899-bc60-4603-8623-0d3107a0eb2a admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:16.969935 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df253821-cf74-433c-82bb-de41d21cb996 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:16.975115 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026d7c47-2312-430e-a256-bd8621f8ad1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:16.975709 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026d7c47-2312-430e-a256-bd8621f8ad1b admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:16.979662 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026d7c47-2312-430e-a256-bd8621f8ad1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:16.979662 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026d7c47-2312-430e-a256-bd8621f8ad1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:16.979662 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026d7c47-2312-430e-a256-bd8621f8ad1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:16.979662 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026d7c47-2312-430e-a256-bd8621f8ad1b admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:16.989651 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 796/1595] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:10:16 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:10:16.990567 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026d7c47-2312-430e-a256-bd8621f8ad1b admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:16.992829 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:10:17.006202 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d49f4e-2824-4ce6-a0da-8ffbae2012c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:17.006754 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d49f4e-2824-4ce6-a0da-8ffbae2012c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:17.007091 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d49f4e-2824-4ce6-a0da-8ffbae2012c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:17.007372 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d49f4e-2824-4ce6-a0da-8ffbae2012c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:17.008009 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d49f4e-2824-4ce6-a0da-8ffbae2012c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:17.009668 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d49f4e-2824-4ce6-a0da-8ffbae2012c4 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:17.020669 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 800/1596] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:10:16 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:10:17.021855 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d49f4e-2824-4ce6-a0da-8ffbae2012c4 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:17.032793 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d49f4e-2824-4ce6-a0da-8ffbae2012c4 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:17.037580 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c2325e-3a25-41eb-aeb0-a10763d96eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:17.038518 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c2325e-3a25-41eb-aeb0-a10763d96eae admin admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:17.038798 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c2325e-3a25-41eb-aeb0-a10763d96eae admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:17.039071 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c2325e-3a25-41eb-aeb0-a10763d96eae admin admin] PATH_INFO: `/v3/projects/dd591617d7554c23b925b2372ead2475/users/69f6e7250ac24151ada8cd4bdde3e070/roles/56e4a96094e9434cbbafb22579c034f5` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:17.046266 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026d7c47-2312-430e-a256-bd8621f8ad1b admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:17.050212 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe108b91-a802-47c8-acdd-cec054dcf945 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:17.050894 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe108b91-a802-47c8-acdd-cec054dcf945 service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:17.051242 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe108b91-a802-47c8-acdd-cec054dcf945 service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:17.051661 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe108b91-a802-47c8-acdd-cec054dcf945 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:17.054486 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c2325e-3a25-41eb-aeb0-a10763d96eae admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd591617d7554c23b925b2372ead2475, user_id=69f6e7250ac24151ada8cd4bdde3e070, role_id=56e4a96094e9434cbbafb22579c034f5)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:17.057188 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c2325e-3a25-41eb-aeb0-a10763d96eae admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:17.060967 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe108b91-a802-47c8-acdd-cec054dcf945 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:17.062479 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe108b91-a802-47c8-acdd-cec054dcf945 service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:17.072064 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 797/1597] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:10:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:10:17.073758 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe108b91-a802-47c8-acdd-cec054dcf945 service neutron] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:17.082301 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe108b91-a802-47c8-acdd-cec054dcf945 service neutron] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:17.087214 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369831f5-4645-4125-baf1-f27c0dbd8aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:17.087806 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369831f5-4645-4125-baf1-f27c0dbd8aa6 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:17.088086 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369831f5-4645-4125-baf1-f27c0dbd8aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:17.088359 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369831f5-4645-4125-baf1-f27c0dbd8aa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:17.089042 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369831f5-4645-4125-baf1-f27c0dbd8aa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:17.090737 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369831f5-4645-4125-baf1-f27c0dbd8aa6 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:17.101676 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 798/1598] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:10:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:10:17.102757 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369831f5-4645-4125-baf1-f27c0dbd8aa6 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:17.112282 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369831f5-4645-4125-baf1-f27c0dbd8aa6 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:17.117060 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e422fa05-45cb-4b38-895e-1ecf4c491c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:17.117674 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e422fa05-45cb-4b38-895e-1ecf4c491c6e admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:17.117952 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e422fa05-45cb-4b38-895e-1ecf4c491c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:17.118224 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e422fa05-45cb-4b38-895e-1ecf4c491c6e admin admin] PATH_INFO: `/v3/projects/8e000f22d8284f6ea2178aa17dc70151/users/dfc4de8c76d0470a892dc8bad4d29441/roles/56e4a96094e9434cbbafb22579c034f5` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:17.127669 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 801/1599] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:10:17 2022] PUT /identity/v3/projects/dd591617d7554c23b925b2372ead2475/users/69f6e7250ac24151ada8cd4bdde3e070/roles/56e4a96094e9434cbbafb22579c034f5 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:10:17.129144 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb9eca-d644-480d-9d70-55efb60bce1a None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:17.129657 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb9eca-d644-480d-9d70-55efb60bce1a None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:17.130030 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb9eca-d644-480d-9d70-55efb60bce1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:17.133877 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e422fa05-45cb-4b38-895e-1ecf4c491c6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e000f22d8284f6ea2178aa17dc70151, user_id=dfc4de8c76d0470a892dc8bad4d29441, role_id=56e4a96094e9434cbbafb22579c034f5)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:17.137187 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e422fa05-45cb-4b38-895e-1ecf4c491c6e admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:17.158802 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-20eb9eca-d644-480d-9d70-55efb60bce1a None None] MFA Rules not processed for user `2625522503744c9993b08b326a6c5e7f`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:17.160016 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-20eb9eca-d644-480d-9d70-55efb60bce1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:17.225121 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 799/1600] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:10:17 2022] PUT /identity/v3/projects/8e000f22d8284f6ea2178aa17dc70151/users/dfc4de8c76d0470a892dc8bad4d29441/roles/56e4a96094e9434cbbafb22579c034f5 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:10:17.225121 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2122e1-7f04-4607-bec4-06fbcbda159a None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:17.225121 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2122e1-7f04-4607-bec4-06fbcbda159a None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:17.225121 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2122e1-7f04-4607-bec4-06fbcbda159a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:17.240222 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 802/1601] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:10:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:10:17.241220 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eb9eca-d644-480d-9d70-55efb60bce1a None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:17.267719 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-0d2122e1-7f04-4607-bec4-06fbcbda159a None None] MFA Rules not processed for user `43c7063420214750b53126a323c85e12`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:17.269080 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-0d2122e1-7f04-4607-bec4-06fbcbda159a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:17.312883 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eb9eca-d644-480d-9d70-55efb60bce1a None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:17.337419 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 800/1602] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:10:17 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:10:17.338151 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2122e1-7f04-4607-bec4-06fbcbda159a None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:17.377022 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:10:17.381225 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:10:17.387777 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d2bcd5-1a4e-4682-bea6-84f7adb4bacf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:17.387777 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d2bcd5-1a4e-4682-bea6-84f7adb4bacf admin admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:17.387777 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d2bcd5-1a4e-4682-bea6-84f7adb4bacf admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:17.387777 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d2bcd5-1a4e-4682-bea6-84f7adb4bacf admin admin] PATH_INFO: `/v3/projects/d85d0d1786b546ffa511708f4634df21/users/d20eabf28f0b40d3ae1e6f2b3b6d6b8c/roles/56e4a96094e9434cbbafb22579c034f5` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:17.405299 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d2bcd5-1a4e-4682-bea6-84f7adb4bacf admin admin] RBAC: Authorizing `identity:create_grant(project_id=d85d0d1786b546ffa511708f4634df21, user_id=d20eabf28f0b40d3ae1e6f2b3b6d6b8c, role_id=56e4a96094e9434cbbafb22579c034f5)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:17.408629 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d2bcd5-1a4e-4682-bea6-84f7adb4bacf admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:17.426962 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2122e1-7f04-4607-bec4-06fbcbda159a None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:17.431213 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff6e983-b74c-48c2-9dc0-8aa3359fa917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:17.431827 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff6e983-b74c-48c2-9dc0-8aa3359fa917 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:17.432106 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff6e983-b74c-48c2-9dc0-8aa3359fa917 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:17.432485 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff6e983-b74c-48c2-9dc0-8aa3359fa917 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:17.433099 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff6e983-b74c-48c2-9dc0-8aa3359fa917 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:17.434748 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff6e983-b74c-48c2-9dc0-8aa3359fa917 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:17.445500 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 801/1603] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:10:17 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:10:17.446372 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff6e983-b74c-48c2-9dc0-8aa3359fa917 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:17.456702 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff6e983-b74c-48c2-9dc0-8aa3359fa917 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:17.461338 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d82ac6-12cb-4ed8-ae5d-9457201d9ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:17.461921 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d82ac6-12cb-4ed8-ae5d-9457201d9ede admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:17.462198 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d82ac6-12cb-4ed8-ae5d-9457201d9ede admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:17.462467 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d82ac6-12cb-4ed8-ae5d-9457201d9ede admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:17.463089 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d82ac6-12cb-4ed8-ae5d-9457201d9ede admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:17.464773 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d82ac6-12cb-4ed8-ae5d-9457201d9ede admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:17.474440 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 802/1604] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:10:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:10:17.475701 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de9de29-aeb1-493e-9a4e-0d53b902f04c None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:17.475989 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de9de29-aeb1-493e-9a4e-0d53b902f04c None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:17.476539 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de9de29-aeb1-493e-9a4e-0d53b902f04c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:17.505313 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-5de9de29-aeb1-493e-9a4e-0d53b902f04c None None] MFA Rules not processed for user `69f6e7250ac24151ada8cd4bdde3e070`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:17.506556 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-5de9de29-aeb1-493e-9a4e-0d53b902f04c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:17.527210 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 803/1605] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:10:17 2022] PUT /identity/v3/projects/d85d0d1786b546ffa511708f4634df21/users/d20eabf28f0b40d3ae1e6f2b3b6d6b8c/roles/56e4a96094e9434cbbafb22579c034f5 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:10:17.528627 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998ef55d-03b0-453c-9b32-297088f09c1e None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:17.529111 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998ef55d-03b0-453c-9b32-297088f09c1e None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:17.529549 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998ef55d-03b0-453c-9b32-297088f09c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:17.548917 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-998ef55d-03b0-453c-9b32-297088f09c1e None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:17.550229 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-998ef55d-03b0-453c-9b32-297088f09c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:17.578026 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 803/1606] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:10:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:10:17.579437 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a83c60-fcf0-4676-84e6-c20782c41d77 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:17.579715 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a83c60-fcf0-4676-84e6-c20782c41d77 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:17.579982 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a83c60-fcf0-4676-84e6-c20782c41d77 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:17.581781 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 804/1607] 10.222.0.18 () {54 vars in 962 bytes} [Sat Aug 6 22:10:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:10:17.582469 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a83c60-fcf0-4676-84e6-c20782c41d77 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:17.617859 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 804/1608] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:10:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:10:17.618726 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998ef55d-03b0-453c-9b32-297088f09c1e None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:17.628992 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998ef55d-03b0-453c-9b32-297088f09c1e None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:17.636530 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5a5b8f-901a-401c-9ca8-786ab5c97a77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:17.636530 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a5b8f-901a-401c-9ca8-786ab5c97a77 admin admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:17.636530 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a5b8f-901a-401c-9ca8-786ab5c97a77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:17.636530 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a5b8f-901a-401c-9ca8-786ab5c97a77 admin admin] PATH_INFO: `/v3/projects/b1386a21549b4faf87836a9be75c670c/users/13f161eb377e4f809b2232c1047c0bb5/roles/56e4a96094e9434cbbafb22579c034f5` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:17.645774 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a83c60-fcf0-4676-84e6-c20782c41d77 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:17.650523 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5a5b8f-901a-401c-9ca8-786ab5c97a77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1386a21549b4faf87836a9be75c670c, user_id=13f161eb377e4f809b2232c1047c0bb5, role_id=56e4a96094e9434cbbafb22579c034f5)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:17.653245 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5a5b8f-901a-401c-9ca8-786ab5c97a77 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:17.658421 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45ac22b-3355-4bee-8bcb-f611a211debb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa71c71de263478da234f3c5716a6e5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:17.659103 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45ac22b-3355-4bee-8bcb-f611a211debb service cinder] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:17.659383 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45ac22b-3355-4bee-8bcb-f611a211debb service cinder] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:17.659655 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45ac22b-3355-4bee-8bcb-f611a211debb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:17.719469 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45ac22b-3355-4bee-8bcb-f611a211debb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:17.720854 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45ac22b-3355-4bee-8bcb-f611a211debb service cinder] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:17.731480 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 805/1609] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:10:17 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:10:17.732345 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45ac22b-3355-4bee-8bcb-f611a211debb service cinder] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:17.742471 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45ac22b-3355-4bee-8bcb-f611a211debb service cinder] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:17.747271 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb843644-c93a-4a05-a336-fa3bb6ebac08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:17.747910 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb843644-c93a-4a05-a336-fa3bb6ebac08 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:17.748184 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb843644-c93a-4a05-a336-fa3bb6ebac08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:17.748477 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb843644-c93a-4a05-a336-fa3bb6ebac08 admin admin] PATH_INFO: `/v3/projects/af0afe3c34994a99a4a27bd3a159420a/users/93438599d0084da0a54ebbae591b4459/roles/56e4a96094e9434cbbafb22579c034f5` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:17.757972 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 805/1610] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:10:17 2022] PUT /identity/v3/projects/b1386a21549b4faf87836a9be75c670c/users/13f161eb377e4f809b2232c1047c0bb5/roles/56e4a96094e9434cbbafb22579c034f5 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:10:17.759300 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1f9fa9-061d-42dd-b3e3-05583fd706b1 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:17.759590 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1f9fa9-061d-42dd-b3e3-05583fd706b1 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:17.759862 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1f9fa9-061d-42dd-b3e3-05583fd706b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:17.768082 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb843644-c93a-4a05-a336-fa3bb6ebac08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af0afe3c34994a99a4a27bd3a159420a, user_id=93438599d0084da0a54ebbae591b4459, role_id=56e4a96094e9434cbbafb22579c034f5)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:17.771014 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb843644-c93a-4a05-a336-fa3bb6ebac08 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:17.788877 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-ca1f9fa9-061d-42dd-b3e3-05583fd706b1 None None] MFA Rules not processed for user `d20eabf28f0b40d3ae1e6f2b3b6d6b8c`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:17.790155 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-ca1f9fa9-061d-42dd-b3e3-05583fd706b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:17.863945 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 806/1611] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:10:17 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:10:17.863945 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7201f546-347f-43e9-a758-113d959dc3e4 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:17.863945 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7201f546-347f-43e9-a758-113d959dc3e4 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:17.863945 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7201f546-347f-43e9-a758-113d959dc3e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:17.877788 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 806/1612] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:10:17 2022] PUT /identity/v3/projects/af0afe3c34994a99a4a27bd3a159420a/users/93438599d0084da0a54ebbae591b4459/roles/56e4a96094e9434cbbafb22579c034f5 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:10:17.879542 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9e3aaf-d3d7-4db6-8324-bef2f1ddcd03 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:17.879825 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9e3aaf-d3d7-4db6-8324-bef2f1ddcd03 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:17.880236 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9e3aaf-d3d7-4db6-8324-bef2f1ddcd03 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:17.882255 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 807/1613] 10.222.0.18 () {54 vars in 962 bytes} [Sat Aug 6 22:10:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:10:17.883122 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9e3aaf-d3d7-4db6-8324-bef2f1ddcd03 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:17.885123 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-ac9e3aaf-d3d7-4db6-8324-bef2f1ddcd03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:17.927307 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-7201f546-347f-43e9-a758-113d959dc3e4 None None] MFA Rules not processed for user `2a1319d33e9144859c6010843be312a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:17.928792 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-7201f546-347f-43e9-a758-113d959dc3e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:17.956669 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9e3aaf-d3d7-4db6-8324-bef2f1ddcd03 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:17.961630 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b343626-5215-4ce3-a901-ba01a6e4b039 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:17.962566 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b343626-5215-4ce3-a901-ba01a6e4b039 service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:17.962843 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b343626-5215-4ce3-a901-ba01a6e4b039 service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:17.963160 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b343626-5215-4ce3-a901-ba01a6e4b039 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:18.029232 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 807/1614] 10.222.0.18 () {58 vars in 1059 bytes} [Sat Aug 6 22:10:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:10:18.030970 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea005be-6597-45e2-a64b-7e8276d3363f None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:18.031446 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea005be-6597-45e2-a64b-7e8276d3363f None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:18.031732 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea005be-6597-45e2-a64b-7e8276d3363f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:18.041320 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b343626-5215-4ce3-a901-ba01a6e4b039 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:18.042751 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b343626-5215-4ce3-a901-ba01a6e4b039 service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:18.050498 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-bea005be-6597-45e2-a64b-7e8276d3363f None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:18.051679 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-bea005be-6597-45e2-a64b-7e8276d3363f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:18.052653 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 808/1615] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:10:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:10:18.053431 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b343626-5215-4ce3-a901-ba01a6e4b039 service neutron] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:18.062822 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b343626-5215-4ce3-a901-ba01a6e4b039 service neutron] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:18.065611 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 808/1616] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:10:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:10:18.068778 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406b11ee-905e-47d0-b2a7-683c1b158b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:18.068778 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b11ee-905e-47d0-b2a7-683c1b158b9c admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:18.068778 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b11ee-905e-47d0-b2a7-683c1b158b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:18.068778 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b11ee-905e-47d0-b2a7-683c1b158b9c admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:18.069396 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406b11ee-905e-47d0-b2a7-683c1b158b9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:18.070986 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69334f00-e76c-4272-9616-68ccf1bd5136 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:18.071432 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406b11ee-905e-47d0-b2a7-683c1b158b9c admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:18.071564 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69334f00-e76c-4272-9616-68ccf1bd5136 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:18.071967 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69334f00-e76c-4272-9616-68ccf1bd5136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:18.105528 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-69334f00-e76c-4272-9616-68ccf1bd5136 None None] MFA Rules not processed for user `93438599d0084da0a54ebbae591b4459`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:18.106846 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-69334f00-e76c-4272-9616-68ccf1bd5136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:18.176832 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:10:18.180365 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 809/1617] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:10:18 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:10:18.181229 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80dd6fd-086e-43c6-be97-b774b99ddc0c None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:18.181632 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80dd6fd-086e-43c6-be97-b774b99ddc0c None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:18.181908 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80dd6fd-086e-43c6-be97-b774b99ddc0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:18.189109 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 809/1618] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:10:18 2022] POST /identity/v3/projects => generated 355 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:10:18.190258 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406b11ee-905e-47d0-b2a7-683c1b158b9c admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:18.203789 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-a80dd6fd-086e-43c6-be97-b774b99ddc0c None None] MFA Rules not processed for user `2a1319d33e9144859c6010843be312a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:18.205098 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-a80dd6fd-086e-43c6-be97-b774b99ddc0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:18.252875 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:10:18.278988 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406b11ee-905e-47d0-b2a7-683c1b158b9c admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:18.288938 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f5c297-8233-407c-b36c-9b484c4491d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a1319d33e9144859c6010843be312a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:18.288938 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f5c297-8233-407c-b36c-9b484c4491d0 service nova] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:18.288938 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f5c297-8233-407c-b36c-9b484c4491d0 service nova] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:18.288938 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f5c297-8233-407c-b36c-9b484c4491d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:18.289476 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 810/1619] 10.222.0.18 () {58 vars in 1059 bytes} [Sat Aug 6 22:10:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:10:18.290302 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80dd6fd-086e-43c6-be97-b774b99ddc0c None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:18.362371 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f5c297-8233-407c-b36c-9b484c4491d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:18.364316 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f5c297-8233-407c-b36c-9b484c4491d0 service nova] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:18.378106 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 810/1620] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:10:18 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:10:18.379023 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f5c297-8233-407c-b36c-9b484c4491d0 service nova] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:18.385152 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80dd6fd-086e-43c6-be97-b774b99ddc0c None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:18.399164 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224b350e-c7a0-4825-804f-c3a404f0b135 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:18.399164 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224b350e-c7a0-4825-804f-c3a404f0b135 service neutron] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:18.399164 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224b350e-c7a0-4825-804f-c3a404f0b135 service neutron] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:18.399164 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224b350e-c7a0-4825-804f-c3a404f0b135 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:18.512700 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224b350e-c7a0-4825-804f-c3a404f0b135 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:18.514506 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224b350e-c7a0-4825-804f-c3a404f0b135 service neutron] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:18.523469 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:10:18.532335 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f5c297-8233-407c-b36c-9b484c4491d0 service nova] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:18.540061 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9148f2ec-51c7-40c3-873f-e0b9eca2c884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:18.540828 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9148f2ec-51c7-40c3-873f-e0b9eca2c884 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:18.541112 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9148f2ec-51c7-40c3-873f-e0b9eca2c884 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:18.541385 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9148f2ec-51c7-40c3-873f-e0b9eca2c884 admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:18.542247 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9148f2ec-51c7-40c3-873f-e0b9eca2c884 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:18.544760 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 811/1621] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:10:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:10:18.545626 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224b350e-c7a0-4825-804f-c3a404f0b135 service neutron] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:18.559434 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9148f2ec-51c7-40c3-873f-e0b9eca2c884 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:18.561426 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224b350e-c7a0-4825-804f-c3a404f0b135 service neutron] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:18.567454 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a938cc-9b99-4b3c-8028-35d51966b331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:18.568231 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a938cc-9b99-4b3c-8028-35d51966b331 admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:18.568562 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a938cc-9b99-4b3c-8028-35d51966b331 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:18.568837 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a938cc-9b99-4b3c-8028-35d51966b331 admin admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:18.569475 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a938cc-9b99-4b3c-8028-35d51966b331 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:18.571672 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a938cc-9b99-4b3c-8028-35d51966b331 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:18.672450 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:10:18.707889 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 812/1622] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:10:18 2022] POST /identity/v3/users => generated 284 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:10:18.708950 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a938cc-9b99-4b3c-8028-35d51966b331 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:18.711860 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 811/1623] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:10:18 2022] POST /identity/v3/projects => generated 363 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:10:18.715848 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9148f2ec-51c7-40c3-873f-e0b9eca2c884 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:18.734440 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a938cc-9b99-4b3c-8028-35d51966b331 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:18.832772 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ba1ba8-d741-413f-95ab-ed31196d2c84 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a1319d33e9144859c6010843be312a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:18.833197 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ba1ba8-d741-413f-95ab-ed31196d2c84 service nova] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:18.833552 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ba1ba8-d741-413f-95ab-ed31196d2c84 service nova] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:18.833866 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ba1ba8-d741-413f-95ab-ed31196d2c84 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:18.858572 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9148f2ec-51c7-40c3-873f-e0b9eca2c884 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:18.869254 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b57f14d-286a-4517-83f3-e806d4087e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:18.869890 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b57f14d-286a-4517-83f3-e806d4087e25 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:18.870176 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b57f14d-286a-4517-83f3-e806d4087e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:18.870455 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b57f14d-286a-4517-83f3-e806d4087e25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:18.871105 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b57f14d-286a-4517-83f3-e806d4087e25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:18.872774 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b57f14d-286a-4517-83f3-e806d4087e25 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:18.893235 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 812/1624] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:10:18 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:10:18.896935 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b57f14d-286a-4517-83f3-e806d4087e25 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:18.903273 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ba1ba8-d741-413f-95ab-ed31196d2c84 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:18.905239 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ba1ba8-d741-413f-95ab-ed31196d2c84 service nova] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:18.916447 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 813/1625] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:10:18 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:10:18.917868 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f67a21-7541-46c4-96e1-e1df03295b1e None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:18.918189 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f67a21-7541-46c4-96e1-e1df03295b1e None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:18.918514 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f67a21-7541-46c4-96e1-e1df03295b1e None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:18.920312 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b57f14d-286a-4517-83f3-e806d4087e25 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:18.920461 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 814/1626] 10.222.0.18 () {54 vars in 912 bytes} [Sat Aug 6 22:10:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:10:18.921170 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f67a21-7541-46c4-96e1-e1df03295b1e None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:18.932835 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bf708d-efac-4cdb-b3f1-a412fb75370d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:18.933456 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bf708d-efac-4cdb-b3f1-a412fb75370d admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:18.933738 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bf708d-efac-4cdb-b3f1-a412fb75370d admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:18.934010 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bf708d-efac-4cdb-b3f1-a412fb75370d admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:18.934642 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bf708d-efac-4cdb-b3f1-a412fb75370d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:18.936617 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bf708d-efac-4cdb-b3f1-a412fb75370d admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:18.938092 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f67a21-7541-46c4-96e1-e1df03295b1e None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:18.944171 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0a47f8-22c6-414e-a4ff-e1083b4f96fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:18.944652 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0a47f8-22c6-414e-a4ff-e1083b4f96fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:18.949783 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0a47f8-22c6-414e-a4ff-e1083b4f96fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:18.949783 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0a47f8-22c6-414e-a4ff-e1083b4f96fc admin admin] PATH_INFO: `/v3/projects/345d71e9fea44171a23fc0b0b93cd2f6/users/8e5caf223bf04e4590f6c226b50113a4/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:18.949783 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:10:18.975545 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:10:18.995555 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0a47f8-22c6-414e-a4ff-e1083b4f96fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=345d71e9fea44171a23fc0b0b93cd2f6, user_id=8e5caf223bf04e4590f6c226b50113a4, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:19.004289 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0a47f8-22c6-414e-a4ff-e1083b4f96fc admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:19.059432 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 813/1627] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:10:18 2022] POST /identity/v3/users => generated 288 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:10:19.064076 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80e2a24-6a75-476f-891f-10baf634f155 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:19.064367 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80e2a24-6a75-476f-891f-10baf634f155 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:19.064690 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80e2a24-6a75-476f-891f-10baf634f155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:19.090746 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 815/1628] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:10:18 2022] PUT /identity/v3/projects/345d71e9fea44171a23fc0b0b93cd2f6/users/8e5caf223bf04e4590f6c226b50113a4/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:10:19.097204 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0a47f8-22c6-414e-a4ff-e1083b4f96fc admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:19.115049 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-a80e2a24-6a75-476f-891f-10baf634f155 None None] MFA Rules not processed for user `2a1319d33e9144859c6010843be312a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:19.116607 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-a80e2a24-6a75-476f-891f-10baf634f155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:19.207234 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0a47f8-22c6-414e-a4ff-e1083b4f96fc admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:19.219175 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 814/1629] 10.222.0.18 () {58 vars in 1009 bytes} [Sat Aug 6 22:10:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:10:19.220393 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80e2a24-6a75-476f-891f-10baf634f155 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:19.221426 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18208a7e-ba26-4465-b0b0-28f06b00e4d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:19.223003 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18208a7e-ba26-4465-b0b0-28f06b00e4d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:19.223283 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18208a7e-ba26-4465-b0b0-28f06b00e4d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:19.223554 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18208a7e-ba26-4465-b0b0-28f06b00e4d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:19.225899 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18208a7e-ba26-4465-b0b0-28f06b00e4d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:19.229833 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18208a7e-ba26-4465-b0b0-28f06b00e4d9 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:19.245250 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80e2a24-6a75-476f-891f-10baf634f155 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:19.254641 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 816/1630] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:10:19 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:10:19.254641 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18208a7e-ba26-4465-b0b0-28f06b00e4d9 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:19.256331 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dcb891-86e2-460e-97ab-aa6a1849c5ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:19.262639 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dcb891-86e2-460e-97ab-aa6a1849c5ec admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:19.262639 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dcb891-86e2-460e-97ab-aa6a1849c5ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:19.262639 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dcb891-86e2-460e-97ab-aa6a1849c5ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:19.262639 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dcb891-86e2-460e-97ab-aa6a1849c5ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:19.262639 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dcb891-86e2-460e-97ab-aa6a1849c5ec admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:19.290259 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 815/1631] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:10:19 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:10:19.291125 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dcb891-86e2-460e-97ab-aa6a1849c5ec admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:19.311627 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dcb891-86e2-460e-97ab-aa6a1849c5ec admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:19.318659 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053333e3-5027-46ed-a794-63a41c7b013c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:19.325521 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053333e3-5027-46ed-a794-63a41c7b013c admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:19.325521 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053333e3-5027-46ed-a794-63a41c7b013c admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:19.325521 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053333e3-5027-46ed-a794-63a41c7b013c admin admin] PATH_INFO: `/v3/projects/1efa895320054043985f5876a5226e21/users/d81ea4d48f924145a6fdd5e73ba05c43/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:19.360968 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053333e3-5027-46ed-a794-63a41c7b013c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1efa895320054043985f5876a5226e21, user_id=d81ea4d48f924145a6fdd5e73ba05c43, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:19.366186 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053333e3-5027-46ed-a794-63a41c7b013c admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:19.378317 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18208a7e-ba26-4465-b0b0-28f06b00e4d9 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:19.382848 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92852e96-93bd-45ec-94bb-25d38ced1761 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa71c71de263478da234f3c5716a6e5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:19.383551 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92852e96-93bd-45ec-94bb-25d38ced1761 service cinder] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:19.383932 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92852e96-93bd-45ec-94bb-25d38ced1761 service cinder] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:19.384311 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92852e96-93bd-45ec-94bb-25d38ced1761 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:19.396857 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92852e96-93bd-45ec-94bb-25d38ced1761 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:19.399006 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92852e96-93bd-45ec-94bb-25d38ced1761 service cinder] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:19.416105 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 817/1632] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:10:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:10:19.417000 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92852e96-93bd-45ec-94bb-25d38ced1761 service cinder] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:19.432692 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92852e96-93bd-45ec-94bb-25d38ced1761 service cinder] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:19.448273 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b6bc79-6fb8-487f-811a-f984cbbee2d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:19.449176 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b6bc79-6fb8-487f-811a-f984cbbee2d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:19.449679 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b6bc79-6fb8-487f-811a-f984cbbee2d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:19.449916 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b6bc79-6fb8-487f-811a-f984cbbee2d6 admin admin] PATH_INFO: `/v3/projects/345d71e9fea44171a23fc0b0b93cd2f6/users/8e5caf223bf04e4590f6c226b50113a4/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:19.452895 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b6bc79-6fb8-487f-811a-f984cbbee2d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=345d71e9fea44171a23fc0b0b93cd2f6, user_id=8e5caf223bf04e4590f6c226b50113a4, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:19.456566 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b6bc79-6fb8-487f-811a-f984cbbee2d6 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:19.481222 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 818/1633] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:10:19 2022] PUT /identity/v3/projects/345d71e9fea44171a23fc0b0b93cd2f6/users/8e5caf223bf04e4590f6c226b50113a4/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:10:19.490378 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faa7907-300b-4e36-93a8-87c9cbb4dcef None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:19.490378 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faa7907-300b-4e36-93a8-87c9cbb4dcef None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:19.490378 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faa7907-300b-4e36-93a8-87c9cbb4dcef None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:19.490627 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 816/1634] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:10:19 2022] PUT /identity/v3/projects/1efa895320054043985f5876a5226e21/users/d81ea4d48f924145a6fdd5e73ba05c43/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:10:19.490718 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 819/1635] 10.222.0.18 () {54 vars in 912 bytes} [Sat Aug 6 22:10:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:10:19.491185 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053333e3-5027-46ed-a794-63a41c7b013c admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:19.492679 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff392d7c-d727-4504-bc72-b45c0f248a74 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:19.493199 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff392d7c-d727-4504-bc72-b45c0f248a74 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:19.493963 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff392d7c-d727-4504-bc72-b45c0f248a74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:19.556178 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-ff392d7c-d727-4504-bc72-b45c0f248a74 None None] MFA Rules not processed for user `8e5caf223bf04e4590f6c226b50113a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:19.557553 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-ff392d7c-d727-4504-bc72-b45c0f248a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:19.622310 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053333e3-5027-46ed-a794-63a41c7b013c admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:19.634908 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19bdf3d-46cd-4774-a846-5208d66d729e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:19.636032 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19bdf3d-46cd-4774-a846-5208d66d729e admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:19.637946 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19bdf3d-46cd-4774-a846-5208d66d729e admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:19.638430 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19bdf3d-46cd-4774-a846-5208d66d729e admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:19.639274 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19bdf3d-46cd-4774-a846-5208d66d729e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:19.643768 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19bdf3d-46cd-4774-a846-5208d66d729e admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:19.673840 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 817/1636] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:10:19 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:10:19.675689 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ee8d9-0f9a-4e9d-a0da-36924fe87ba2 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:19.676061 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ee8d9-0f9a-4e9d-a0da-36924fe87ba2 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:19.677286 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ee8d9-0f9a-4e9d-a0da-36924fe87ba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:19.682783 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:10:19.694853 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 820/1637] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:10:19 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:10:19.696333 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff392d7c-d727-4504-bc72-b45c0f248a74 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:19.709057 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-052ee8d9-0f9a-4e9d-a0da-36924fe87ba2 None None] MFA Rules not processed for user `2a1319d33e9144859c6010843be312a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:19.713982 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff392d7c-d727-4504-bc72-b45c0f248a74 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:19.714271 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-052ee8d9-0f9a-4e9d-a0da-36924fe87ba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:19.717697 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8fc5ed-6627-49d4-85a3-54739f3e2f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:19.719123 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8fc5ed-6627-49d4-85a3-54739f3e2f48 admin admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:19.719460 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8fc5ed-6627-49d4-85a3-54739f3e2f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:19.719722 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8fc5ed-6627-49d4-85a3-54739f3e2f48 admin admin] PATH_INFO: `/v3/projects/1efa895320054043985f5876a5226e21/users/d81ea4d48f924145a6fdd5e73ba05c43/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:19.724003 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8fc5ed-6627-49d4-85a3-54739f3e2f48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1efa895320054043985f5876a5226e21, user_id=d81ea4d48f924145a6fdd5e73ba05c43, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:19.727132 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8fc5ed-6627-49d4-85a3-54739f3e2f48 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:19.738684 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 821/1638] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:10:19 2022] PUT /identity/v3/projects/1efa895320054043985f5876a5226e21/users/d81ea4d48f924145a6fdd5e73ba05c43/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:10:19.739657 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8fc5ed-6627-49d4-85a3-54739f3e2f48 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:19.768091 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:10:19.796969 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 818/1639] 10.222.0.18 () {58 vars in 1009 bytes} [Sat Aug 6 22:10:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:10:19.798593 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3198e437-aef9-4ff4-8c04-017d0aa66816 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:19.798880 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3198e437-aef9-4ff4-8c04-017d0aa66816 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:19.799149 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3198e437-aef9-4ff4-8c04-017d0aa66816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:19.833245 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-3198e437-aef9-4ff4-8c04-017d0aa66816 None None] MFA Rules not processed for user `d81ea4d48f924145a6fdd5e73ba05c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:19.835307 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-3198e437-aef9-4ff4-8c04-017d0aa66816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:19.850736 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8fc5ed-6627-49d4-85a3-54739f3e2f48 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:19.863205 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc32d0d-a369-4f66-a5a4-a08221b1c80a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa71c71de263478da234f3c5716a6e5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:19.863827 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc32d0d-a369-4f66-a5a4-a08221b1c80a service cinder] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:19.864443 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc32d0d-a369-4f66-a5a4-a08221b1c80a service cinder] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:19.864443 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc32d0d-a369-4f66-a5a4-a08221b1c80a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:19.988660 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc32d0d-a369-4f66-a5a4-a08221b1c80a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:19.991608 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc32d0d-a369-4f66-a5a4-a08221b1c80a service cinder] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:20.006951 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 822/1640] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:10:19 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:10:20.016140 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 819/1641] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:10:19 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:10:20.597585 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc32d0d-a369-4f66-a5a4-a08221b1c80a service cinder] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:20.597585 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc32d0d-a369-4f66-a5a4-a08221b1c80a service cinder] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:20.597585 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edb5e90-6f8c-4fea-9a83-18d835cbd133 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa71c71de263478da234f3c5716a6e5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:20.597585 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edb5e90-6f8c-4fea-9a83-18d835cbd133 service cinder] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:20.608296 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edb5e90-6f8c-4fea-9a83-18d835cbd133 service cinder] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:20.608296 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edb5e90-6f8c-4fea-9a83-18d835cbd133 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:20.608296 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edb5e90-6f8c-4fea-9a83-18d835cbd133 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:20.608296 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edb5e90-6f8c-4fea-9a83-18d835cbd133 service cinder] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:20.608296 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 823/1642] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:10:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:10:20.613839 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3198e437-aef9-4ff4-8c04-017d0aa66816 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:20.613839 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3198e437-aef9-4ff4-8c04-017d0aa66816 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:20.613839 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e845aba9-c231-4c34-88d5-00e00a5e1e95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa71c71de263478da234f3c5716a6e5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:20.613839 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e845aba9-c231-4c34-88d5-00e00a5e1e95 service cinder] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:20.617617 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e845aba9-c231-4c34-88d5-00e00a5e1e95 service cinder] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:20.617617 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e845aba9-c231-4c34-88d5-00e00a5e1e95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:20.617617 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e845aba9-c231-4c34-88d5-00e00a5e1e95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:20.617617 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e845aba9-c231-4c34-88d5-00e00a5e1e95 service cinder] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:20.617617 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 820/1643] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:10:20 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:10:20.741188 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edb5e90-6f8c-4fea-9a83-18d835cbd133 service cinder] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:20.753855 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edb5e90-6f8c-4fea-9a83-18d835cbd133 service cinder] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:20.758222 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b42d2c-da00-461b-a68f-4cbd7ac497ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa71c71de263478da234f3c5716a6e5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:20.758621 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b42d2c-da00-461b-a68f-4cbd7ac497ba service cinder] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:20.758947 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b42d2c-da00-461b-a68f-4cbd7ac497ba service cinder] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:20.759223 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b42d2c-da00-461b-a68f-4cbd7ac497ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:20.773075 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:10:20.853719 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e845aba9-c231-4c34-88d5-00e00a5e1e95 service cinder] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:20.909061 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b42d2c-da00-461b-a68f-4cbd7ac497ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:20.910883 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b42d2c-da00-461b-a68f-4cbd7ac497ba service cinder] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:20.941600 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 824/1644] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:10:20 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:10:20.949467 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e845aba9-c231-4c34-88d5-00e00a5e1e95 service cinder] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:20.962802 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3817e44-146f-4b6e-ba0b-1b53a0b83565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:20.963723 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3817e44-146f-4b6e-ba0b-1b53a0b83565 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:20.964248 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3817e44-146f-4b6e-ba0b-1b53a0b83565 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:20.964642 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3817e44-146f-4b6e-ba0b-1b53a0b83565 admin admin] PATH_INFO: `/v3/users/69f6e7250ac24151ada8cd4bdde3e070` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:20.967449 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3817e44-146f-4b6e-ba0b-1b53a0b83565 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69f6e7250ac24151ada8cd4bdde3e070)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:20.969242 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: WARNING py.warnings [None req-c3817e44-146f-4b6e-ba0b-1b53a0b83565 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 06 22:10:20.969242 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: warnings.warn(msg) Aug 06 22:10:20.969242 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: Aug 06 22:10:20.970598 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3817e44-146f-4b6e-ba0b-1b53a0b83565 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:21.334585 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-c3817e44-146f-4b6e-ba0b-1b53a0b83565 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69f6e7250ac24151ada8cd4bdde3e070'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:21.356744 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:10:21.389245 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-c3817e44-146f-4b6e-ba0b-1b53a0b83565 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69f6e7250ac24151ada8cd4bdde3e070'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:21.416947 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-c3817e44-146f-4b6e-ba0b-1b53a0b83565 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69f6e7250ac24151ada8cd4bdde3e070'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:21.416947 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-c3817e44-146f-4b6e-ba0b-1b53a0b83565 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69f6e7250ac24151ada8cd4bdde3e070'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:21.419510 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-c3817e44-146f-4b6e-ba0b-1b53a0b83565 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69f6e7250ac24151ada8cd4bdde3e070'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:21.565687 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 821/1645] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:10:20 2022] DELETE /identity/v3/users/69f6e7250ac24151ada8cd4bdde3e070 => generated 0 bytes in 713 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 22:10:21.570093 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6511f8be-a615-49dd-93ef-86eff05660c0 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:21.570543 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6511f8be-a615-49dd-93ef-86eff05660c0 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:21.571031 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6511f8be-a615-49dd-93ef-86eff05660c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:21.658346 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-6511f8be-a615-49dd-93ef-86eff05660c0 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:21.689477 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-6511f8be-a615-49dd-93ef-86eff05660c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:21.806937 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 825/1646] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:10:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:10:21.829254 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3817e44-146f-4b6e-ba0b-1b53a0b83565 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:21.830909 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-c3817e44-146f-4b6e-ba0b-1b53a0b83565 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:21.975290 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3817e44-146f-4b6e-ba0b-1b53a0b83565 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:21.994795 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2743bb-5f55-4265-85b8-42b95413bdbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:21.996107 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2743bb-5f55-4265-85b8-42b95413bdbf service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:21.996425 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2743bb-5f55-4265-85b8-42b95413bdbf service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:21.996708 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2743bb-5f55-4265-85b8-42b95413bdbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:22.018946 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2743bb-5f55-4265-85b8-42b95413bdbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:22.020696 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2743bb-5f55-4265-85b8-42b95413bdbf service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:22.045477 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 822/1647] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:10:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:10:22.902903 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6511f8be-a615-49dd-93ef-86eff05660c0 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:22.904546 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-6511f8be-a615-49dd-93ef-86eff05660c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:22.976934 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6511f8be-a615-49dd-93ef-86eff05660c0 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:22.986282 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc76e64-3753-4329-a0ab-0753ccde5142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:22.987028 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc76e64-3753-4329-a0ab-0753ccde5142 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:22.987312 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc76e64-3753-4329-a0ab-0753ccde5142 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:22.987587 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc76e64-3753-4329-a0ab-0753ccde5142 admin admin] PATH_INFO: `/v3/projects/dd591617d7554c23b925b2372ead2475` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:22.989224 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc76e64-3753-4329-a0ab-0753ccde5142 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd591617d7554c23b925b2372ead2475)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:22.990314 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: WARNING py.warnings [None req-1fc76e64-3753-4329-a0ab-0753ccde5142 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 06 22:10:22.990314 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: warnings.warn(msg) Aug 06 22:10:22.990314 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: Aug 06 22:10:22.995284 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc76e64-3753-4329-a0ab-0753ccde5142 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:23.087480 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2743bb-5f55-4265-85b8-42b95413bdbf service neutron] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:23.089918 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-4a2743bb-5f55-4265-85b8-42b95413bdbf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:23.126233 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2743bb-5f55-4265-85b8-42b95413bdbf service neutron] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:23.161692 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-1fc76e64-3753-4329-a0ab-0753ccde5142 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd591617d7554c23b925b2372ead2475', 'request_id': 'req-1fc76e64-3753-4329-a0ab-0753ccde5142'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:23.223749 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434406df-b893-48a1-a1ad-f190d509ed99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:23.224793 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434406df-b893-48a1-a1ad-f190d509ed99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:23.225257 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434406df-b893-48a1-a1ad-f190d509ed99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:23.225663 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434406df-b893-48a1-a1ad-f190d509ed99 admin admin] PATH_INFO: `/v3/users/93438599d0084da0a54ebbae591b4459` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:23.231681 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434406df-b893-48a1-a1ad-f190d509ed99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93438599d0084da0a54ebbae591b4459)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:23.231681 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434406df-b893-48a1-a1ad-f190d509ed99 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:23.340515 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-1fc76e64-3753-4329-a0ab-0753ccde5142 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd591617d7554c23b925b2372ead2475', 'request_id': 'req-1fc76e64-3753-4329-a0ab-0753ccde5142'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:23.349482 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-1fc76e64-3753-4329-a0ab-0753ccde5142 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd591617d7554c23b925b2372ead2475', 'request_id': 'req-1fc76e64-3753-4329-a0ab-0753ccde5142'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:23.377885 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-1fc76e64-3753-4329-a0ab-0753ccde5142 admin admin] The token cache is being invalidate because project dd591617d7554c23b925b2372ead2475 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:10:23.378373 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-1fc76e64-3753-4329-a0ab-0753ccde5142 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:23.380615 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-1fc76e64-3753-4329-a0ab-0753ccde5142 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:23.384614 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 826/1648] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:10:22 2022] DELETE /identity/v3/projects/dd591617d7554c23b925b2372ead2475 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:10:23.386138 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3c9832-ea57-4e17-8632-4595a2688a72 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:23.386680 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3c9832-ea57-4e17-8632-4595a2688a72 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:23.387143 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3c9832-ea57-4e17-8632-4595a2688a72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:23.423545 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-dd3c9832-ea57-4e17-8632-4595a2688a72 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:23.424856 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-dd3c9832-ea57-4e17-8632-4595a2688a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:23.435107 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 827/1649] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:10:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:10:23.444269 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588e124c-760b-444a-a6fe-b1b4602226e7 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:23.444659 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588e124c-760b-444a-a6fe-b1b4602226e7 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:23.445009 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588e124c-760b-444a-a6fe-b1b4602226e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:23.472086 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-588e124c-760b-444a-a6fe-b1b4602226e7 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:23.474851 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-588e124c-760b-444a-a6fe-b1b4602226e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:23.487622 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 828/1650] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:10:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 22:10:23.491968 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588e124c-760b-444a-a6fe-b1b4602226e7 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:23.494439 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-434406df-b893-48a1-a1ad-f190d509ed99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93438599d0084da0a54ebbae591b4459'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:23.508128 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588e124c-760b-444a-a6fe-b1b4602226e7 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:23.517022 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-434406df-b893-48a1-a1ad-f190d509ed99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93438599d0084da0a54ebbae591b4459'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:23.535186 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-434406df-b893-48a1-a1ad-f190d509ed99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93438599d0084da0a54ebbae591b4459'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:23.535924 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-434406df-b893-48a1-a1ad-f190d509ed99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93438599d0084da0a54ebbae591b4459'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:23.536037 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655fe5d3-3d9a-437e-a7ef-70dd1b8d56a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:23.539866 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-434406df-b893-48a1-a1ad-f190d509ed99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93438599d0084da0a54ebbae591b4459'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:23.539992 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655fe5d3-3d9a-437e-a7ef-70dd1b8d56a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:23.539992 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655fe5d3-3d9a-437e-a7ef-70dd1b8d56a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:23.540135 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:10:23.540213 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655fe5d3-3d9a-437e-a7ef-70dd1b8d56a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:23.540687 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655fe5d3-3d9a-437e-a7ef-70dd1b8d56a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 22:10:23.541067 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655fe5d3-3d9a-437e-a7ef-70dd1b8d56a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:23.543496 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655fe5d3-3d9a-437e-a7ef-70dd1b8d56a5 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:23.565039 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 829/1651] 10.222.0.18 () {58 vars in 1153 bytes} [Sat Aug 6 22:10:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 22:10:23.569877 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655fe5d3-3d9a-437e-a7ef-70dd1b8d56a5 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:23.572991 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-655fe5d3-3d9a-437e-a7ef-70dd1b8d56a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:23.593718 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655fe5d3-3d9a-437e-a7ef-70dd1b8d56a5 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:23.599470 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc05b9db-b39f-4258-ad32-1d8952629289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:23.600526 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc05b9db-b39f-4258-ad32-1d8952629289 admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:23.600526 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc05b9db-b39f-4258-ad32-1d8952629289 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:23.600686 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc05b9db-b39f-4258-ad32-1d8952629289 admin admin] PATH_INFO: `/v3/projects` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:23.601595 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc05b9db-b39f-4258-ad32-1d8952629289 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:23.603816 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc05b9db-b39f-4258-ad32-1d8952629289 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:23.620774 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:10:23.642396 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 823/1652] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:10:23 2022] DELETE /identity/v3/users/93438599d0084da0a54ebbae591b4459 => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:10:23.646874 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aacd3d-5745-4adf-b326-e7276ceb48a8 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:23.647157 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aacd3d-5745-4adf-b326-e7276ceb48a8 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:23.647433 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aacd3d-5745-4adf-b326-e7276ceb48a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:23.679660 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-f9aacd3d-5745-4adf-b326-e7276ceb48a8 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:23.681136 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-f9aacd3d-5745-4adf-b326-e7276ceb48a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:23.736627 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 830/1653] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:10:23 2022] POST /identity/v3/projects => generated 361 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:10:23.740677 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc05b9db-b39f-4258-ad32-1d8952629289 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:23.836083 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 824/1654] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:10:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:10:23.846383 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aacd3d-5745-4adf-b326-e7276ceb48a8 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:23.847949 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-f9aacd3d-5745-4adf-b326-e7276ceb48a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:23.893669 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc05b9db-b39f-4258-ad32-1d8952629289 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:23.908767 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d997e20-8a60-4d8f-959c-eaaeea72a713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:23.909389 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d997e20-8a60-4d8f-959c-eaaeea72a713 admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:23.909995 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d997e20-8a60-4d8f-959c-eaaeea72a713 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:23.909995 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d997e20-8a60-4d8f-959c-eaaeea72a713 admin admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:23.910716 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d997e20-8a60-4d8f-959c-eaaeea72a713 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:23.912698 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d997e20-8a60-4d8f-959c-eaaeea72a713 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:23.980063 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aacd3d-5745-4adf-b326-e7276ceb48a8 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:24.039267 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 831/1655] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:10:23 2022] POST /identity/v3/users => generated 281 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:10:24.043508 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d997e20-8a60-4d8f-959c-eaaeea72a713 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:24.065508 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d997e20-8a60-4d8f-959c-eaaeea72a713 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:24.071153 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98a0c83-6f50-4e4b-b7fa-3eaa9b5070b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:24.072104 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98a0c83-6f50-4e4b-b7fa-3eaa9b5070b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:24.073677 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98a0c83-6f50-4e4b-b7fa-3eaa9b5070b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:24.073677 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98a0c83-6f50-4e4b-b7fa-3eaa9b5070b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:24.077949 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9d2955-dedd-445e-a18b-203adc1ec718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:24.078628 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d2955-dedd-445e-a18b-203adc1ec718 admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:24.079076 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d2955-dedd-445e-a18b-203adc1ec718 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:24.079466 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d2955-dedd-445e-a18b-203adc1ec718 admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:24.080253 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9d2955-dedd-445e-a18b-203adc1ec718 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:24.082235 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9d2955-dedd-445e-a18b-203adc1ec718 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:24.086250 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98a0c83-6f50-4e4b-b7fa-3eaa9b5070b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:24.088952 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98a0c83-6f50-4e4b-b7fa-3eaa9b5070b9 service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:24.098118 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 832/1656] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:10:24 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 22:10:24.103172 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9d2955-dedd-445e-a18b-203adc1ec718 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:24.104902 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 825/1657] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:10:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 22:10:24.115800 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9d2955-dedd-445e-a18b-203adc1ec718 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:24.122614 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45ea640-854a-4552-8841-8d357dc3fcd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:24.123424 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45ea640-854a-4552-8841-8d357dc3fcd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:24.123843 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45ea640-854a-4552-8841-8d357dc3fcd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:24.124252 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45ea640-854a-4552-8841-8d357dc3fcd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:24.126817 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45ea640-854a-4552-8841-8d357dc3fcd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:24.128618 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45ea640-854a-4552-8841-8d357dc3fcd4 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:24.149250 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 833/1658] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:10:24 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:10:24.154353 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98a0c83-6f50-4e4b-b7fa-3eaa9b5070b9 service neutron] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:24.160983 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98a0c83-6f50-4e4b-b7fa-3eaa9b5070b9 service neutron] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:24.167105 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4cd595-16a0-47c9-b00d-999c122638fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:24.168157 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4cd595-16a0-47c9-b00d-999c122638fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:24.168654 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4cd595-16a0-47c9-b00d-999c122638fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:24.168986 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4cd595-16a0-47c9-b00d-999c122638fe admin admin] PATH_INFO: `/v3/projects/b582eb3e54e54444943891b7ea6b3ee4/users/0a82743949de49f0b32192615c403f84/roles/56e4a96094e9434cbbafb22579c034f5` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:24.196943 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4cd595-16a0-47c9-b00d-999c122638fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=b582eb3e54e54444943891b7ea6b3ee4, user_id=0a82743949de49f0b32192615c403f84, role_id=56e4a96094e9434cbbafb22579c034f5)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:24.202094 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4cd595-16a0-47c9-b00d-999c122638fe admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:24.320747 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f7179a-2448-4901-bd7e-3508a8e36e65 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:24.321100 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f7179a-2448-4901-bd7e-3508a8e36e65 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:24.322314 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f7179a-2448-4901-bd7e-3508a8e36e65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:24.330715 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 826/1659] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:10:24 2022] PUT /identity/v3/projects/b582eb3e54e54444943891b7ea6b3ee4/users/0a82743949de49f0b32192615c403f84/roles/56e4a96094e9434cbbafb22579c034f5 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 22:10:24.337940 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4cd595-16a0-47c9-b00d-999c122638fe admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:24.345904 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-fd4cd595-16a0-47c9-b00d-999c122638fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:24.442213 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-07f7179a-2448-4901-bd7e-3508a8e36e65 None None] MFA Rules not processed for user `0a82743949de49f0b32192615c403f84`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:24.443982 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-07f7179a-2448-4901-bd7e-3508a8e36e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:24.618569 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4cd595-16a0-47c9-b00d-999c122638fe admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:24.626122 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6365bb84-8410-4d00-9d33-fec83c54b6fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:24.626929 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6365bb84-8410-4d00-9d33-fec83c54b6fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:24.627251 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6365bb84-8410-4d00-9d33-fec83c54b6fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:24.627639 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6365bb84-8410-4d00-9d33-fec83c54b6fa admin admin] PATH_INFO: `/v3/users/2625522503744c9993b08b326a6c5e7f` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:24.630306 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6365bb84-8410-4d00-9d33-fec83c54b6fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=2625522503744c9993b08b326a6c5e7f)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:24.632613 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6365bb84-8410-4d00-9d33-fec83c54b6fa admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:24.684794 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 834/1660] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:10:24 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 367 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:10:24.697009 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f7179a-2448-4901-bd7e-3508a8e36e65 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:24.712841 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f7179a-2448-4901-bd7e-3508a8e36e65 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:24.723720 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ccfcf8-7d11-4dd1-9d03-8163397c619a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:24.724582 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ccfcf8-7d11-4dd1-9d03-8163397c619a admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:24.724999 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ccfcf8-7d11-4dd1-9d03-8163397c619a admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:24.725411 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ccfcf8-7d11-4dd1-9d03-8163397c619a admin admin] PATH_INFO: `/v3/projects` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:24.726676 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ccfcf8-7d11-4dd1-9d03-8163397c619a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:24.729023 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ccfcf8-7d11-4dd1-9d03-8163397c619a admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:24.830297 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 835/1661] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:10:24 2022] POST /identity/v3/projects => generated 365 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:10:24.834315 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ccfcf8-7d11-4dd1-9d03-8163397c619a admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:24.911258 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-6365bb84-8410-4d00-9d33-fec83c54b6fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2625522503744c9993b08b326a6c5e7f'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:24.968669 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-6365bb84-8410-4d00-9d33-fec83c54b6fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2625522503744c9993b08b326a6c5e7f'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:25.003092 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ccfcf8-7d11-4dd1-9d03-8163397c619a admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:25.012062 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-6365bb84-8410-4d00-9d33-fec83c54b6fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2625522503744c9993b08b326a6c5e7f'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:25.013528 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-6365bb84-8410-4d00-9d33-fec83c54b6fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2625522503744c9993b08b326a6c5e7f'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:25.016827 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-6365bb84-8410-4d00-9d33-fec83c54b6fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2625522503744c9993b08b326a6c5e7f'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:25.020826 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e7b81d-f34b-4b6b-b47e-13dbcee93e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:25.021558 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e7b81d-f34b-4b6b-b47e-13dbcee93e63 admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:25.021846 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e7b81d-f34b-4b6b-b47e-13dbcee93e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:25.022122 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e7b81d-f34b-4b6b-b47e-13dbcee93e63 admin admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:25.026224 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e7b81d-f34b-4b6b-b47e-13dbcee93e63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:25.031202 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e7b81d-f34b-4b6b-b47e-13dbcee93e63 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:25.117652 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 836/1662] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:10:24 2022] POST /identity/v3/users => generated 289 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:10:25.122172 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 827/1663] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:10:24 2022] DELETE /identity/v3/users/2625522503744c9993b08b326a6c5e7f => generated 0 bytes in 787 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:10:25.123851 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6365bb84-8410-4d00-9d33-fec83c54b6fa admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:25.123965 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e7b81d-f34b-4b6b-b47e-13dbcee93e63 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:25.128376 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-c4e7b81d-f34b-4b6b-b47e-13dbcee93e63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:25.128512 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-6365bb84-8410-4d00-9d33-fec83c54b6fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:25.160792 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e7b81d-f34b-4b6b-b47e-13dbcee93e63 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:25.181412 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:INSERT=2,keystone:DELETE=1,keystone:SELECT=97 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:10:25.293779 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc6917a-6585-4ca7-b5c5-bc506721e2aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:25.294765 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc6917a-6585-4ca7-b5c5-bc506721e2aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:25.295433 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc6917a-6585-4ca7-b5c5-bc506721e2aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:25.295925 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc6917a-6585-4ca7-b5c5-bc506721e2aa admin admin] PATH_INFO: `/v3/projects/af0afe3c34994a99a4a27bd3a159420a` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:25.300713 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc6917a-6585-4ca7-b5c5-bc506721e2aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=af0afe3c34994a99a4a27bd3a159420a)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:25.304544 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc6917a-6585-4ca7-b5c5-bc506721e2aa admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:25.309194 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:10:25.326209 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6365bb84-8410-4d00-9d33-fec83c54b6fa admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:25.353713 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4414b50-d4c9-49fa-9340-f7a6b64b9566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:25.355816 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4414b50-d4c9-49fa-9340-f7a6b64b9566 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:25.355816 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4414b50-d4c9-49fa-9340-f7a6b64b9566 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:25.355816 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4414b50-d4c9-49fa-9340-f7a6b64b9566 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:25.356714 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4414b50-d4c9-49fa-9340-f7a6b64b9566 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:25.359474 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4414b50-d4c9-49fa-9340-f7a6b64b9566 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:25.390431 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 828/1664] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:10:25 2022] GET /identity/v3/roles => generated 1420 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:10:25.392123 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab6c08a-b04c-41c0-8334-db5c0ec03fb9 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:25.392427 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab6c08a-b04c-41c0-8334-db5c0ec03fb9 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:25.395652 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab6c08a-b04c-41c0-8334-db5c0ec03fb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:25.450814 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-9ab6c08a-b04c-41c0-8334-db5c0ec03fb9 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:25.452065 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-9ab6c08a-b04c-41c0-8334-db5c0ec03fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:25.486668 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 829/1665] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:10:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:10:25.488394 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab6c08a-b04c-41c0-8334-db5c0ec03fb9 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:25.514950 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab6c08a-b04c-41c0-8334-db5c0ec03fb9 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:25.524946 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-3bc6917a-6585-4ca7-b5c5-bc506721e2aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af0afe3c34994a99a4a27bd3a159420a', 'request_id': 'req-3bc6917a-6585-4ca7-b5c5-bc506721e2aa'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:25.599773 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-3bc6917a-6585-4ca7-b5c5-bc506721e2aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af0afe3c34994a99a4a27bd3a159420a', 'request_id': 'req-3bc6917a-6585-4ca7-b5c5-bc506721e2aa'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:25.605402 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-3bc6917a-6585-4ca7-b5c5-bc506721e2aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af0afe3c34994a99a4a27bd3a159420a', 'request_id': 'req-3bc6917a-6585-4ca7-b5c5-bc506721e2aa'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:25.633969 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-3bc6917a-6585-4ca7-b5c5-bc506721e2aa admin admin] The token cache is being invalidate because project af0afe3c34994a99a4a27bd3a159420a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:10:25.635590 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-3bc6917a-6585-4ca7-b5c5-bc506721e2aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:25.637242 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-3bc6917a-6585-4ca7-b5c5-bc506721e2aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:25.644864 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 837/1666] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:10:25 2022] DELETE /identity/v3/projects/af0afe3c34994a99a4a27bd3a159420a => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:10:25.654084 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc6917a-6585-4ca7-b5c5-bc506721e2aa admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:25.654084 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-3bc6917a-6585-4ca7-b5c5-bc506721e2aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:25.663802 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6ec285-086f-4e27-98fc-708453d5e887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:25.664759 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6ec285-086f-4e27-98fc-708453d5e887 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:25.665242 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6ec285-086f-4e27-98fc-708453d5e887 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:25.665716 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6ec285-086f-4e27-98fc-708453d5e887 admin admin] PATH_INFO: `/v3/projects/50c98ea9dbb345b7a198f03686b52e85/users/5285d561c8104df5a33452bcfb75c931/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:25.692943 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6ec285-086f-4e27-98fc-708453d5e887 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50c98ea9dbb345b7a198f03686b52e85, user_id=5285d561c8104df5a33452bcfb75c931, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:25.694736 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6ec285-086f-4e27-98fc-708453d5e887 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:25.757204 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc6917a-6585-4ca7-b5c5-bc506721e2aa admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:25.761378 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b68235-6303-4988-ac34-b2b6c82bc96c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:25.762054 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b68235-6303-4988-ac34-b2b6c82bc96c service neutron] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:25.762329 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b68235-6303-4988-ac34-b2b6c82bc96c service neutron] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:25.762596 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b68235-6303-4988-ac34-b2b6c82bc96c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:25.765501 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-b2b68235-6303-4988-ac34-b2b6c82bc96c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:25.783315 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b68235-6303-4988-ac34-b2b6c82bc96c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:25.784941 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b68235-6303-4988-ac34-b2b6c82bc96c service neutron] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:25.796033 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 830/1667] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:10:25 2022] PUT /identity/v3/projects/50c98ea9dbb345b7a198f03686b52e85/users/5285d561c8104df5a33452bcfb75c931/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:10:25.798186 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6ec285-086f-4e27-98fc-708453d5e887 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:25.802629 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-2a6ec285-086f-4e27-98fc-708453d5e887 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:25.807698 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 838/1668] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:10:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:10:25.808955 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b68235-6303-4988-ac34-b2b6c82bc96c service neutron] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:25.810613 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-b2b68235-6303-4988-ac34-b2b6c82bc96c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:25.823578 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b68235-6303-4988-ac34-b2b6c82bc96c service neutron] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:25.935377 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da321419-07d1-43d0-9aa3-4f3d583a850e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:25.935998 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da321419-07d1-43d0-9aa3-4f3d583a850e admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:25.936894 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da321419-07d1-43d0-9aa3-4f3d583a850e admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:25.937179 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da321419-07d1-43d0-9aa3-4f3d583a850e admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:25.937855 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da321419-07d1-43d0-9aa3-4f3d583a850e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:25.939599 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da321419-07d1-43d0-9aa3-4f3d583a850e admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:25.963713 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6ec285-086f-4e27-98fc-708453d5e887 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:25.964711 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 839/1669] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:10:25 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:10:25.966444 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da321419-07d1-43d0-9aa3-4f3d583a850e admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:25.976854 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395715d3-1ec2-4c4c-acda-6ad0957d8066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:25.977529 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395715d3-1ec2-4c4c-acda-6ad0957d8066 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:25.978007 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395715d3-1ec2-4c4c-acda-6ad0957d8066 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:25.978107 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395715d3-1ec2-4c4c-acda-6ad0957d8066 admin admin] PATH_INFO: `/v3/users/d81ea4d48f924145a6fdd5e73ba05c43` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:25.979823 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395715d3-1ec2-4c4c-acda-6ad0957d8066 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d81ea4d48f924145a6fdd5e73ba05c43)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:25.982414 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395715d3-1ec2-4c4c-acda-6ad0957d8066 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:25.988728 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da321419-07d1-43d0-9aa3-4f3d583a850e admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:26.005711 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca93e84-c60f-4a80-bd11-1efd76b1fd4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:26.006454 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca93e84-c60f-4a80-bd11-1efd76b1fd4a admin admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:26.007028 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca93e84-c60f-4a80-bd11-1efd76b1fd4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:26.007028 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca93e84-c60f-4a80-bd11-1efd76b1fd4a admin admin] PATH_INFO: `/v3/projects/50c98ea9dbb345b7a198f03686b52e85/users/5285d561c8104df5a33452bcfb75c931/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:26.011037 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca93e84-c60f-4a80-bd11-1efd76b1fd4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=50c98ea9dbb345b7a198f03686b52e85, user_id=5285d561c8104df5a33452bcfb75c931, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:26.021838 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca93e84-c60f-4a80-bd11-1efd76b1fd4a admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:26.038796 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 840/1670] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:10:25 2022] PUT /identity/v3/projects/50c98ea9dbb345b7a198f03686b52e85/users/5285d561c8104df5a33452bcfb75c931/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:10:26.051090 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e58018-0303-49dc-bd33-a0c06de27f3f None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:26.051377 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e58018-0303-49dc-bd33-a0c06de27f3f None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:26.051669 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e58018-0303-49dc-bd33-a0c06de27f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:26.150714 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-d9e58018-0303-49dc-bd33-a0c06de27f3f None None] MFA Rules not processed for user `5285d561c8104df5a33452bcfb75c931`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:26.153027 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-d9e58018-0303-49dc-bd33-a0c06de27f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:26.219983 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-395715d3-1ec2-4c4c-acda-6ad0957d8066 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd81ea4d48f924145a6fdd5e73ba05c43'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:26.253382 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-395715d3-1ec2-4c4c-acda-6ad0957d8066 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd81ea4d48f924145a6fdd5e73ba05c43'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:26.278801 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-395715d3-1ec2-4c4c-acda-6ad0957d8066 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd81ea4d48f924145a6fdd5e73ba05c43'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:26.280290 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-395715d3-1ec2-4c4c-acda-6ad0957d8066 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd81ea4d48f924145a6fdd5e73ba05c43'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:26.282549 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-395715d3-1ec2-4c4c-acda-6ad0957d8066 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd81ea4d48f924145a6fdd5e73ba05c43'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:26.364923 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 831/1671] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:10:25 2022] DELETE /identity/v3/users/d81ea4d48f924145a6fdd5e73ba05c43 => generated 0 bytes in 567 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:10:26.382752 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 841/1672] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:10:26 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 22:10:26.393588 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395715d3-1ec2-4c4c-acda-6ad0957d8066 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:26.396263 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-395715d3-1ec2-4c4c-acda-6ad0957d8066 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:26.574606 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395715d3-1ec2-4c4c-acda-6ad0957d8066 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:26.578688 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eac3f8-7be3-4763-9038-6176c715dc41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa71c71de263478da234f3c5716a6e5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:26.579311 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eac3f8-7be3-4763-9038-6176c715dc41 service cinder] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:26.579589 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eac3f8-7be3-4763-9038-6176c715dc41 service cinder] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:26.579862 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eac3f8-7be3-4763-9038-6176c715dc41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:26.601635 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e58018-0303-49dc-bd33-a0c06de27f3f None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:26.603422 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-d9e58018-0303-49dc-bd33-a0c06de27f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:26.758611 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eac3f8-7be3-4763-9038-6176c715dc41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:26.760141 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eac3f8-7be3-4763-9038-6176c715dc41 service cinder] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:26.776477 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 832/1673] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:10:26 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:10:26.820295 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e58018-0303-49dc-bd33-a0c06de27f3f None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:26.893210 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:10:26.958060 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa54aef-d12b-4785-9220-9fc4e25f33c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:26.958924 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa54aef-d12b-4785-9220-9fc4e25f33c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:26.959303 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa54aef-d12b-4785-9220-9fc4e25f33c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:26.959676 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa54aef-d12b-4785-9220-9fc4e25f33c7 admin admin] PATH_INFO: `/v3/projects/2d9aaf79afbe4f87a96eae7bb7d6b8e3` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:26.961595 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa54aef-d12b-4785-9220-9fc4e25f33c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d9aaf79afbe4f87a96eae7bb7d6b8e3)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:26.966928 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa54aef-d12b-4785-9220-9fc4e25f33c7 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:27.086398 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eac3f8-7be3-4763-9038-6176c715dc41 service cinder] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:27.096722 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eac3f8-7be3-4763-9038-6176c715dc41 service cinder] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:27.101894 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3669d8-b2a3-476d-a6cb-cf92a058ef73 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa71c71de263478da234f3c5716a6e5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:27.102775 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3669d8-b2a3-476d-a6cb-cf92a058ef73 service cinder] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:27.103376 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3669d8-b2a3-476d-a6cb-cf92a058ef73 service cinder] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:27.104498 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3669d8-b2a3-476d-a6cb-cf92a058ef73 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:27.127585 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-0c3669d8-b2a3-476d-a6cb-cf92a058ef73 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:27.164431 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-5fa54aef-d12b-4785-9220-9fc4e25f33c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d9aaf79afbe4f87a96eae7bb7d6b8e3', 'request_id': 'req-5fa54aef-d12b-4785-9220-9fc4e25f33c7'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:27.248943 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-5fa54aef-d12b-4785-9220-9fc4e25f33c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d9aaf79afbe4f87a96eae7bb7d6b8e3', 'request_id': 'req-5fa54aef-d12b-4785-9220-9fc4e25f33c7'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:27.258406 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-5fa54aef-d12b-4785-9220-9fc4e25f33c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d9aaf79afbe4f87a96eae7bb7d6b8e3', 'request_id': 'req-5fa54aef-d12b-4785-9220-9fc4e25f33c7'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:27.268692 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3669d8-b2a3-476d-a6cb-cf92a058ef73 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:27.270272 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3669d8-b2a3-476d-a6cb-cf92a058ef73 service cinder] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:27.284319 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-5fa54aef-d12b-4785-9220-9fc4e25f33c7 admin admin] The token cache is being invalidate because project 2d9aaf79afbe4f87a96eae7bb7d6b8e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:10:27.285951 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-5fa54aef-d12b-4785-9220-9fc4e25f33c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:27.287624 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-5fa54aef-d12b-4785-9220-9fc4e25f33c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:27.292905 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 833/1675] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:10:27 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 22:10:27.293159 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 842/1675] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:10:26 2022] DELETE /identity/v3/projects/2d9aaf79afbe4f87a96eae7bb7d6b8e3 => generated 0 bytes in 690 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:10:27.298054 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1ce543-dc43-49a5-9726-458a59fbcebe None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:27.298608 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1ce543-dc43-49a5-9726-458a59fbcebe None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:27.299027 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1ce543-dc43-49a5-9726-458a59fbcebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:27.354439 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-fe1ce543-dc43-49a5-9726-458a59fbcebe None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:27.356373 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-fe1ce543-dc43-49a5-9726-458a59fbcebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:27.506590 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 834/1676] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:10:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:10:27.520925 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e852f59-3aef-4bd1-84ba-7cccda20ef02 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:27.521243 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e852f59-3aef-4bd1-84ba-7cccda20ef02 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:27.521507 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e852f59-3aef-4bd1-84ba-7cccda20ef02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:27.573929 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-1e852f59-3aef-4bd1-84ba-7cccda20ef02 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:27.576822 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-1e852f59-3aef-4bd1-84ba-7cccda20ef02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:27.601640 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 843/1677] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:10:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:10:27.604095 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:10:27.607877 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1ce543-dc43-49a5-9726-458a59fbcebe None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:27.620720 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1ce543-dc43-49a5-9726-458a59fbcebe None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:27.649752 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2affb9-d040-4cbe-a1dc-b8980413ba09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:27.650522 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2affb9-d040-4cbe-a1dc-b8980413ba09 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:27.651279 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2affb9-d040-4cbe-a1dc-b8980413ba09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:27.652359 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2affb9-d040-4cbe-a1dc-b8980413ba09 admin admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:27.653148 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2affb9-d040-4cbe-a1dc-b8980413ba09 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 22:10:27.654467 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2affb9-d040-4cbe-a1dc-b8980413ba09 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:27.659542 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2affb9-d040-4cbe-a1dc-b8980413ba09 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:27.697135 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 835/1678] 10.222.0.18 () {58 vars in 1153 bytes} [Sat Aug 6 22:10:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 22:10:27.699277 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e852f59-3aef-4bd1-84ba-7cccda20ef02 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:27.711109 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e852f59-3aef-4bd1-84ba-7cccda20ef02 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:27.725820 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afdca79-d1a1-488c-8832-f907a7f7c02b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:27.726998 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afdca79-d1a1-488c-8832-f907a7f7c02b admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:27.727380 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afdca79-d1a1-488c-8832-f907a7f7c02b admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:27.727734 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afdca79-d1a1-488c-8832-f907a7f7c02b admin admin] PATH_INFO: `/v3/projects` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:27.740650 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afdca79-d1a1-488c-8832-f907a7f7c02b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:27.747395 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afdca79-d1a1-488c-8832-f907a7f7c02b admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:27.906352 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 844/1679] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:10:27 2022] POST /identity/v3/projects => generated 381 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:10:27.908742 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2affb9-d040-4cbe-a1dc-b8980413ba09 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:27.927943 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2affb9-d040-4cbe-a1dc-b8980413ba09 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:27.967044 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afdca79-d1a1-488c-8832-f907a7f7c02b admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:27.969807 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-3afdca79-d1a1-488c-8832-f907a7f7c02b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:28.147368 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ea4712-dbbc-497a-92de-2b739a2887a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:28.148821 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ea4712-dbbc-497a-92de-2b739a2887a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:28.149207 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ea4712-dbbc-497a-92de-2b739a2887a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:28.149887 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ea4712-dbbc-497a-92de-2b739a2887a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:28.150677 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ea4712-dbbc-497a-92de-2b739a2887a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:28.153910 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ea4712-dbbc-497a-92de-2b739a2887a8 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:28.175829 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afdca79-d1a1-488c-8832-f907a7f7c02b admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:28.186026 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3645109-3446-43a9-8c0e-9d5e38fba31a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:28.187343 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3645109-3446-43a9-8c0e-9d5e38fba31a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:28.189420 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3645109-3446-43a9-8c0e-9d5e38fba31a admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:28.189920 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3645109-3446-43a9-8c0e-9d5e38fba31a admin admin] PATH_INFO: `/v3/projects/1efa895320054043985f5876a5226e21` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:28.208837 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3645109-3446-43a9-8c0e-9d5e38fba31a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1efa895320054043985f5876a5226e21)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:28.213993 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3645109-3446-43a9-8c0e-9d5e38fba31a admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:28.279164 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 836/1680] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:10:27 2022] POST /identity/v3/users => generated 291 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:10:28.282101 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ea4712-dbbc-497a-92de-2b739a2887a8 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:28.307384 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ea4712-dbbc-497a-92de-2b739a2887a8 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:28.320252 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654ad619-164c-4bc3-ad03-e51313b594e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:28.321131 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654ad619-164c-4bc3-ad03-e51313b594e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:28.321556 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654ad619-164c-4bc3-ad03-e51313b594e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:28.321958 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654ad619-164c-4bc3-ad03-e51313b594e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:28.323441 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654ad619-164c-4bc3-ad03-e51313b594e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:28.325404 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654ad619-164c-4bc3-ad03-e51313b594e2 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:28.351418 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 837/1681] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:10:28 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:10:28.354765 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654ad619-164c-4bc3-ad03-e51313b594e2 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:28.394721 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-a3645109-3446-43a9-8c0e-9d5e38fba31a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1efa895320054043985f5876a5226e21', 'request_id': 'req-a3645109-3446-43a9-8c0e-9d5e38fba31a'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:28.407638 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654ad619-164c-4bc3-ad03-e51313b594e2 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:28.499587 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-a3645109-3446-43a9-8c0e-9d5e38fba31a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1efa895320054043985f5876a5226e21', 'request_id': 'req-a3645109-3446-43a9-8c0e-9d5e38fba31a'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:28.506160 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-a3645109-3446-43a9-8c0e-9d5e38fba31a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1efa895320054043985f5876a5226e21', 'request_id': 'req-a3645109-3446-43a9-8c0e-9d5e38fba31a'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:28.509455 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cf536d-2fd4-4714-b29a-b12b64dab72d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:28.510077 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cf536d-2fd4-4714-b29a-b12b64dab72d admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:28.510356 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cf536d-2fd4-4714-b29a-b12b64dab72d admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:28.510622 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cf536d-2fd4-4714-b29a-b12b64dab72d admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:28.511265 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cf536d-2fd4-4714-b29a-b12b64dab72d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:28.512888 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cf536d-2fd4-4714-b29a-b12b64dab72d admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:28.530101 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-a3645109-3446-43a9-8c0e-9d5e38fba31a admin admin] The token cache is being invalidate because project 1efa895320054043985f5876a5226e21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:10:28.531117 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-a3645109-3446-43a9-8c0e-9d5e38fba31a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:28.531320 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 838/1682] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:10:28 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:10:28.536170 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-a3645109-3446-43a9-8c0e-9d5e38fba31a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:28.540756 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cf536d-2fd4-4714-b29a-b12b64dab72d admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:28.540846 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 845/1683] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:10:27 2022] DELETE /identity/v3/projects/1efa895320054043985f5876a5226e21 => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:10:28.540971 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-84cf536d-2fd4-4714-b29a-b12b64dab72d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:28.544024 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d447852-3c3f-4981-b698-4f380493a25e None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:28.544659 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d447852-3c3f-4981-b698-4f380493a25e None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:28.545046 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d447852-3c3f-4981-b698-4f380493a25e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:28.576344 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cf536d-2fd4-4714-b29a-b12b64dab72d admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:28.582598 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-8d447852-3c3f-4981-b698-4f380493a25e None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:28.583650 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b887f825-c867-4ed7-827d-2bf4648e2660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:28.583796 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-8d447852-3c3f-4981-b698-4f380493a25e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:28.585026 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b887f825-c867-4ed7-827d-2bf4648e2660 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:28.585524 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b887f825-c867-4ed7-827d-2bf4648e2660 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:28.587676 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b887f825-c867-4ed7-827d-2bf4648e2660 admin admin] PATH_INFO: `/v3/projects/a2bea300a56e4e3a86080740b5c2daa7/users/8750d883fc8747a29dc1051cfaa0c5f6/roles/56e4a96094e9434cbbafb22579c034f5` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:28.607031 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 846/1684] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:10:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 22:10:28.615353 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd0fcd5-86df-4ce4-bce8-2a25b11b71f1 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:28.615632 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd0fcd5-86df-4ce4-bce8-2a25b11b71f1 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:28.616052 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd0fcd5-86df-4ce4-bce8-2a25b11b71f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:28.618385 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b887f825-c867-4ed7-827d-2bf4648e2660 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2bea300a56e4e3a86080740b5c2daa7, user_id=8750d883fc8747a29dc1051cfaa0c5f6, role_id=56e4a96094e9434cbbafb22579c034f5)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:28.623028 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b887f825-c867-4ed7-827d-2bf4648e2660 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:28.645908 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-1dd0fcd5-86df-4ce4-bce8-2a25b11b71f1 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:28.647347 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-1dd0fcd5-86df-4ce4-bce8-2a25b11b71f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:28.659036 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 847/1685] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:10:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 22:10:28.662987 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd0fcd5-86df-4ce4-bce8-2a25b11b71f1 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:28.676691 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd0fcd5-86df-4ce4-bce8-2a25b11b71f1 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:28.682408 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95399920-d26a-45eb-9815-31031bd856d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:28.683372 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95399920-d26a-45eb-9815-31031bd856d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:28.684230 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95399920-d26a-45eb-9815-31031bd856d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:28.685003 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95399920-d26a-45eb-9815-31031bd856d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:28.685676 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95399920-d26a-45eb-9815-31031bd856d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 22:10:28.686255 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95399920-d26a-45eb-9815-31031bd856d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:28.688593 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95399920-d26a-45eb-9815-31031bd856d7 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:28.708798 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 848/1686] 10.222.0.18 () {58 vars in 1153 bytes} [Sat Aug 6 22:10:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 22:10:28.710869 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95399920-d26a-45eb-9815-31031bd856d7 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:28.730706 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 839/1687] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:10:28 2022] PUT /identity/v3/projects/a2bea300a56e4e3a86080740b5c2daa7/users/8750d883fc8747a29dc1051cfaa0c5f6/roles/56e4a96094e9434cbbafb22579c034f5 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:10:28.743619 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efd10ec-b2e1-4074-b311-e5a9d2099a10 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:28.744068 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efd10ec-b2e1-4074-b311-e5a9d2099a10 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:28.744477 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efd10ec-b2e1-4074-b311-e5a9d2099a10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:28.750475 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95399920-d26a-45eb-9815-31031bd856d7 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:28.790614 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:10:28.799737 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-0efd10ec-b2e1-4074-b311-e5a9d2099a10 None None] MFA Rules not processed for user `8750d883fc8747a29dc1051cfaa0c5f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:28.802164 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-0efd10ec-b2e1-4074-b311-e5a9d2099a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:28.871570 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a49ec20-477c-41f5-b4e8-5c1cd84384a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:28.872448 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a49ec20-477c-41f5-b4e8-5c1cd84384a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:28.872562 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a49ec20-477c-41f5-b4e8-5c1cd84384a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:28.873233 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a49ec20-477c-41f5-b4e8-5c1cd84384a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:28.875358 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a49ec20-477c-41f5-b4e8-5c1cd84384a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:28.879103 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a49ec20-477c-41f5-b4e8-5c1cd84384a3 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:29.045178 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 849/1688] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:10:28 2022] POST /identity/v3/projects => generated 393 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:10:29.047271 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f838935f-cb02-4d0a-bd9c-2a6a1b9ff77d None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:29.047562 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f838935f-cb02-4d0a-bd9c-2a6a1b9ff77d None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:29.047839 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f838935f-cb02-4d0a-bd9c-2a6a1b9ff77d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:29.084281 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 840/1689] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:10:28 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:10:29.085491 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efd10ec-b2e1-4074-b311-e5a9d2099a10 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:29.108921 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-f838935f-cb02-4d0a-bd9c-2a6a1b9ff77d None None] MFA Rules not processed for user `0d6e35038fed4b44b986902a5ef55ce8`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:29.110545 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-f838935f-cb02-4d0a-bd9c-2a6a1b9ff77d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:29.119117 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efd10ec-b2e1-4074-b311-e5a9d2099a10 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:29.141919 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:10:29.199725 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 850/1690] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:10:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:10:29.201224 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f838935f-cb02-4d0a-bd9c-2a6a1b9ff77d None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:29.288248 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef78d9fc-ea8f-4757-940a-be02801a3b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:29.289875 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef78d9fc-ea8f-4757-940a-be02801a3b8c admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:29.290321 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef78d9fc-ea8f-4757-940a-be02801a3b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:29.296742 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef78d9fc-ea8f-4757-940a-be02801a3b8c admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:29.296827 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f838935f-cb02-4d0a-bd9c-2a6a1b9ff77d None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:29.296897 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef78d9fc-ea8f-4757-940a-be02801a3b8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:29.296897 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef78d9fc-ea8f-4757-940a-be02801a3b8c admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:29.297155 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11200e7e-4654-46c9-b676-ee8c572354db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:29.297942 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11200e7e-4654-46c9-b676-ee8c572354db admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:29.298463 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11200e7e-4654-46c9-b676-ee8c572354db admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:29.298812 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11200e7e-4654-46c9-b676-ee8c572354db admin admin] PATH_INFO: `/v3/projects` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:29.299517 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11200e7e-4654-46c9-b676-ee8c572354db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:29.301904 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11200e7e-4654-46c9-b676-ee8c572354db admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:29.389503 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 841/1691] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:10:29 2022] POST /identity/v3/users => generated 297 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:10:29.391292 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef78d9fc-ea8f-4757-940a-be02801a3b8c admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:29.393210 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-ef78d9fc-ea8f-4757-940a-be02801a3b8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:29.430898 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 851/1692] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:10:29 2022] POST /identity/v3/projects => generated 381 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:10:29.432634 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11200e7e-4654-46c9-b676-ee8c572354db admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:29.573459 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef78d9fc-ea8f-4757-940a-be02801a3b8c admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:29.580110 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11200e7e-4654-46c9-b676-ee8c572354db admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:29.597142 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40a532a-e75f-41e4-b988-c5eda0ae4f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:29.598209 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40a532a-e75f-41e4-b988-c5eda0ae4f35 admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:29.598209 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40a532a-e75f-41e4-b988-c5eda0ae4f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:29.600324 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40a532a-e75f-41e4-b988-c5eda0ae4f35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:29.601687 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40a532a-e75f-41e4-b988-c5eda0ae4f35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:29.605647 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40a532a-e75f-41e4-b988-c5eda0ae4f35 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:29.641193 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 852/1693] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:10:29 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:10:29.641193 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40a532a-e75f-41e4-b988-c5eda0ae4f35 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:29.673014 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40a532a-e75f-41e4-b988-c5eda0ae4f35 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:29.679146 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf0a936-7165-4d61-9c1b-03ac4637963e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:29.680814 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf0a936-7165-4d61-9c1b-03ac4637963e admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:29.680814 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf0a936-7165-4d61-9c1b-03ac4637963e admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:29.681311 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf0a936-7165-4d61-9c1b-03ac4637963e admin admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:29.682600 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf0a936-7165-4d61-9c1b-03ac4637963e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:29.686339 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf0a936-7165-4d61-9c1b-03ac4637963e admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:29.747447 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb62b2c-5e5d-45e4-a354-c174ee41f814 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a1319d33e9144859c6010843be312a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:29.749065 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb62b2c-5e5d-45e4-a354-c174ee41f814 service nova] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:29.749398 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb62b2c-5e5d-45e4-a354-c174ee41f814 service nova] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:29.749673 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb62b2c-5e5d-45e4-a354-c174ee41f814 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:29.832309 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 853/1694] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:10:29 2022] POST /identity/v3/users => generated 297 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:10:29.834362 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf0a936-7165-4d61-9c1b-03ac4637963e admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:29.867381 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf0a936-7165-4d61-9c1b-03ac4637963e admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:29.881063 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9c2247-84db-4200-a040-e09f841937eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:29.882247 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9c2247-84db-4200-a040-e09f841937eb admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:29.882719 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9c2247-84db-4200-a040-e09f841937eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:29.883083 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9c2247-84db-4200-a040-e09f841937eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:29.883793 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9c2247-84db-4200-a040-e09f841937eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:29.885628 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9c2247-84db-4200-a040-e09f841937eb admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:29.899026 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 854/1695] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:10:29 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:10:29.899833 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb62b2c-5e5d-45e4-a354-c174ee41f814 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:29.900203 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9c2247-84db-4200-a040-e09f841937eb admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:29.902113 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb62b2c-5e5d-45e4-a354-c174ee41f814 service nova] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:29.932221 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 842/1696] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:10:29 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:10:29.934684 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb62b2c-5e5d-45e4-a354-c174ee41f814 service nova] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:29.944928 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9c2247-84db-4200-a040-e09f841937eb admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:29.962117 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c319f01-7bcb-41f5-a6a7-303b86fd93a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:29.963573 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c319f01-7bcb-41f5-a6a7-303b86fd93a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:29.963854 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c319f01-7bcb-41f5-a6a7-303b86fd93a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:29.965961 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb62b2c-5e5d-45e4-a354-c174ee41f814 service nova] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:29.968035 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c319f01-7bcb-41f5-a6a7-303b86fd93a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:29.969513 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c319f01-7bcb-41f5-a6a7-303b86fd93a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:29.979483 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d152be66-e092-4535-9e65-41279c476cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:29.980782 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d152be66-e092-4535-9e65-41279c476cf1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:29.981688 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c319f01-7bcb-41f5-a6a7-303b86fd93a5 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:29.982086 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d152be66-e092-4535-9e65-41279c476cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:29.984122 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d152be66-e092-4535-9e65-41279c476cf1 admin admin] PATH_INFO: `/v3/projects/96359b60fff44193864fab8bd71cfed8/users/3c37a7f2f79e4a4183a626fe6f6b5834/roles/56e4a96094e9434cbbafb22579c034f5` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:30.026894 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 855/1697] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:10:29 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:10:30.035152 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c319f01-7bcb-41f5-a6a7-303b86fd93a5 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:30.046414 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d152be66-e092-4535-9e65-41279c476cf1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96359b60fff44193864fab8bd71cfed8, user_id=3c37a7f2f79e4a4183a626fe6f6b5834, role_id=56e4a96094e9434cbbafb22579c034f5)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:30.065231 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:10:30.070153 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d152be66-e092-4535-9e65-41279c476cf1 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:30.071120 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c319f01-7bcb-41f5-a6a7-303b86fd93a5 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:30.088360 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f86f390-fa63-46bf-83f0-375b938a78db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:30.094672 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f86f390-fa63-46bf-83f0-375b938a78db admin admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:30.095140 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f86f390-fa63-46bf-83f0-375b938a78db admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:30.095426 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f86f390-fa63-46bf-83f0-375b938a78db admin admin] PATH_INFO: `/v3/projects/42fddb3a3b3444e9bf84bf38ec12168e/users/b6d1bef15904498ea8f678e510d441e0/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:30.160381 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f86f390-fa63-46bf-83f0-375b938a78db admin admin] RBAC: Authorizing `identity:create_grant(project_id=42fddb3a3b3444e9bf84bf38ec12168e, user_id=b6d1bef15904498ea8f678e510d441e0, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:30.160833 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 843/1698] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:10:29 2022] PUT /identity/v3/projects/96359b60fff44193864fab8bd71cfed8/users/3c37a7f2f79e4a4183a626fe6f6b5834/roles/56e4a96094e9434cbbafb22579c034f5 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:10:30.164731 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64167e05-0bf2-445c-a9a9-4b063e8da8c9 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:30.165013 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64167e05-0bf2-445c-a9a9-4b063e8da8c9 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:30.165312 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64167e05-0bf2-445c-a9a9-4b063e8da8c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:30.168128 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f86f390-fa63-46bf-83f0-375b938a78db admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:30.232597 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-64167e05-0bf2-445c-a9a9-4b063e8da8c9 None None] MFA Rules not processed for user `3c37a7f2f79e4a4183a626fe6f6b5834`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:30.234686 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-64167e05-0bf2-445c-a9a9-4b063e8da8c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:30.322510 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 856/1699] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:10:30 2022] PUT /identity/v3/projects/42fddb3a3b3444e9bf84bf38ec12168e/users/b6d1bef15904498ea8f678e510d441e0/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:10:30.326187 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f86f390-fa63-46bf-83f0-375b938a78db admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:30.424690 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 844/1700] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:10:30 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:10:30.435487 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64167e05-0bf2-445c-a9a9-4b063e8da8c9 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:30.512867 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f86f390-fa63-46bf-83f0-375b938a78db admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:30.532861 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec90ca4e-cee1-41a5-8266-fddd1266f382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:30.533605 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec90ca4e-cee1-41a5-8266-fddd1266f382 admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:30.533889 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec90ca4e-cee1-41a5-8266-fddd1266f382 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:30.534160 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec90ca4e-cee1-41a5-8266-fddd1266f382 admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:30.534928 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec90ca4e-cee1-41a5-8266-fddd1266f382 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:30.539096 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec90ca4e-cee1-41a5-8266-fddd1266f382 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:30.556170 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 857/1701] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:10:30 2022] GET /identity/v3/roles => generated 1420 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:10:30.560704 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec90ca4e-cee1-41a5-8266-fddd1266f382 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:30.575172 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec90ca4e-cee1-41a5-8266-fddd1266f382 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:30.587597 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41af483e-446e-4f3f-b93d-3611c040e8f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:30.588192 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41af483e-446e-4f3f-b93d-3611c040e8f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:30.588559 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41af483e-446e-4f3f-b93d-3611c040e8f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:30.588945 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41af483e-446e-4f3f-b93d-3611c040e8f5 admin admin] PATH_INFO: `/v3/projects/42fddb3a3b3444e9bf84bf38ec12168e/users/b6d1bef15904498ea8f678e510d441e0/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:30.590421 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64167e05-0bf2-445c-a9a9-4b063e8da8c9 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:30.600395 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41af483e-446e-4f3f-b93d-3611c040e8f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42fddb3a3b3444e9bf84bf38ec12168e, user_id=b6d1bef15904498ea8f678e510d441e0, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:30.600395 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41af483e-446e-4f3f-b93d-3611c040e8f5 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:30.601064 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd14326-3efc-4603-8b7b-86be2f39b1ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:30.601064 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd14326-3efc-4603-8b7b-86be2f39b1ce admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:30.602582 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd14326-3efc-4603-8b7b-86be2f39b1ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:30.602886 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd14326-3efc-4603-8b7b-86be2f39b1ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:30.603548 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd14326-3efc-4603-8b7b-86be2f39b1ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:30.612871 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd14326-3efc-4603-8b7b-86be2f39b1ce admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:30.628351 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 858/1702] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:10:30 2022] PUT /identity/v3/projects/42fddb3a3b3444e9bf84bf38ec12168e/users/b6d1bef15904498ea8f678e510d441e0/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:10:30.632728 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa87c53c-9d0e-4b10-b148-6c1872c47f9e None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:30.633539 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa87c53c-9d0e-4b10-b148-6c1872c47f9e None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:30.633826 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa87c53c-9d0e-4b10-b148-6c1872c47f9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:30.749395 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-fa87c53c-9d0e-4b10-b148-6c1872c47f9e None None] MFA Rules not processed for user `b6d1bef15904498ea8f678e510d441e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:30.752482 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-fa87c53c-9d0e-4b10-b148-6c1872c47f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:30.779664 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:10:30.792275 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 845/1703] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:10:30 2022] POST /identity/v3/projects => generated 393 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 22:10:30.793981 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd14326-3efc-4603-8b7b-86be2f39b1ce admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:30.883942 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 859/1704] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:10:30 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:10:30.891695 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa87c53c-9d0e-4b10-b148-6c1872c47f9e None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:30.894088 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-fa87c53c-9d0e-4b10-b148-6c1872c47f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:30.953666 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd14326-3efc-4603-8b7b-86be2f39b1ce admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:30.966660 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a2bb54-2196-44d8-83de-65218b0b90f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:30.966859 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a2bb54-2196-44d8-83de-65218b0b90f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:30.967149 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a2bb54-2196-44d8-83de-65218b0b90f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:30.967906 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a2bb54-2196-44d8-83de-65218b0b90f2 admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:30.968208 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a2bb54-2196-44d8-83de-65218b0b90f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:30.970674 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a2bb54-2196-44d8-83de-65218b0b90f2 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:31.002492 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa87c53c-9d0e-4b10-b148-6c1872c47f9e None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:31.068555 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 846/1705] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:10:30 2022] POST /identity/v3/users => generated 303 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:10:31.069863 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3052fab7-9c3d-4d51-a53d-288fb2e57dce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa71c71de263478da234f3c5716a6e5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:31.072863 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3052fab7-9c3d-4d51-a53d-288fb2e57dce service cinder] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:31.072947 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3052fab7-9c3d-4d51-a53d-288fb2e57dce service cinder] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:31.073339 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a2bb54-2196-44d8-83de-65218b0b90f2 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:31.073615 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3052fab7-9c3d-4d51-a53d-288fb2e57dce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:31.115885 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a2bb54-2196-44d8-83de-65218b0b90f2 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:31.136082 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04305ee-fa11-472e-b866-401910384769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:31.136752 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04305ee-fa11-472e-b866-401910384769 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:31.137036 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04305ee-fa11-472e-b866-401910384769 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:31.137314 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04305ee-fa11-472e-b866-401910384769 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:31.138412 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04305ee-fa11-472e-b866-401910384769 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:31.144446 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04305ee-fa11-472e-b866-401910384769 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:31.161275 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3052fab7-9c3d-4d51-a53d-288fb2e57dce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:31.162435 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3052fab7-9c3d-4d51-a53d-288fb2e57dce service cinder] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:31.179335 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 860/1706] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:10:30 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 22:10:31.192868 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 847/1707] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:10:31 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:10:31.193881 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3052fab7-9c3d-4d51-a53d-288fb2e57dce service cinder] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:31.214367 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3052fab7-9c3d-4d51-a53d-288fb2e57dce service cinder] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:31.233095 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b78361-c41a-41f3-84b3-c8b617b4c790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:31.234838 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b78361-c41a-41f3-84b3-c8b617b4c790 admin admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:31.235225 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b78361-c41a-41f3-84b3-c8b617b4c790 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:31.236201 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b78361-c41a-41f3-84b3-c8b617b4c790 admin admin] PATH_INFO: `/v3/projects/50b87f329654425597e3b41eea4e3dda/users/da5ce139eab449eb88d0018feb9a9060/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:31.271654 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b78361-c41a-41f3-84b3-c8b617b4c790 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50b87f329654425597e3b41eea4e3dda, user_id=da5ce139eab449eb88d0018feb9a9060, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:31.274709 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b78361-c41a-41f3-84b3-c8b617b4c790 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:31.335495 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 861/1708] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:10:31 2022] PUT /identity/v3/projects/50b87f329654425597e3b41eea4e3dda/users/da5ce139eab449eb88d0018feb9a9060/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:10:31.339012 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04305ee-fa11-472e-b866-401910384769 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:31.372136 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04305ee-fa11-472e-b866-401910384769 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:31.485401 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b78361-c41a-41f3-84b3-c8b617b4c790 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:31.527123 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d861a6-7864-4c3c-94cb-45717f39e659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:31.527123 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d861a6-7864-4c3c-94cb-45717f39e659 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:31.527713 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d861a6-7864-4c3c-94cb-45717f39e659 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:31.527713 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d861a6-7864-4c3c-94cb-45717f39e659 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:31.528322 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d861a6-7864-4c3c-94cb-45717f39e659 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:31.538172 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d861a6-7864-4c3c-94cb-45717f39e659 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:31.569547 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 848/1709] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:10:31 2022] GET /identity/v3/roles => generated 1420 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:10:31.571923 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d861a6-7864-4c3c-94cb-45717f39e659 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:31.590179 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:10:31.601967 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d861a6-7864-4c3c-94cb-45717f39e659 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:31.624297 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36650ab3-665b-49f4-a366-b12e941c3c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:31.625248 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36650ab3-665b-49f4-a366-b12e941c3c73 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:31.625675 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36650ab3-665b-49f4-a366-b12e941c3c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:31.626057 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36650ab3-665b-49f4-a366-b12e941c3c73 admin admin] PATH_INFO: `/v3/projects/50b87f329654425597e3b41eea4e3dda/users/da5ce139eab449eb88d0018feb9a9060/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:31.629501 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36650ab3-665b-49f4-a366-b12e941c3c73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50b87f329654425597e3b41eea4e3dda, user_id=da5ce139eab449eb88d0018feb9a9060, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:31.633301 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36650ab3-665b-49f4-a366-b12e941c3c73 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:31.653155 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b78361-c41a-41f3-84b3-c8b617b4c790 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:31.661548 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0479c7-f653-4e3c-b468-52b64f754e97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa71c71de263478da234f3c5716a6e5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:31.665368 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0479c7-f653-4e3c-b468-52b64f754e97 service cinder] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:31.665662 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0479c7-f653-4e3c-b468-52b64f754e97 service cinder] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:31.665936 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0479c7-f653-4e3c-b468-52b64f754e97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:31.691682 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 849/1710] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:10:31 2022] PUT /identity/v3/projects/50b87f329654425597e3b41eea4e3dda/users/da5ce139eab449eb88d0018feb9a9060/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:10:31.696134 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcefaec-2890-4030-8653-737f152751c5 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:31.698046 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcefaec-2890-4030-8653-737f152751c5 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:31.699244 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcefaec-2890-4030-8653-737f152751c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:31.783537 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-efcefaec-2890-4030-8653-737f152751c5 None None] MFA Rules not processed for user `da5ce139eab449eb88d0018feb9a9060`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:31.785123 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-efcefaec-2890-4030-8653-737f152751c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:31.820379 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0479c7-f653-4e3c-b468-52b64f754e97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:31.822436 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0479c7-f653-4e3c-b468-52b64f754e97 service cinder] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:31.859605 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 862/1711] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:10:31 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:10:31.976992 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 850/1712] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:10:31 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:10:32.044389 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c867bdf-d185-4d49-ba4b-5c934cfb3ba4 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:32.044863 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c867bdf-d185-4d49-ba4b-5c934cfb3ba4 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:32.045360 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c867bdf-d185-4d49-ba4b-5c934cfb3ba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:32.092234 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-6c867bdf-d185-4d49-ba4b-5c934cfb3ba4 None None] MFA Rules not processed for user `dfc4de8c76d0470a892dc8bad4d29441`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:32.093686 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-6c867bdf-d185-4d49-ba4b-5c934cfb3ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:32.247787 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 863/1713] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:10:32 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:10:32.260335 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:10:32.265991 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcefaec-2890-4030-8653-737f152751c5 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:32.267493 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-efcefaec-2890-4030-8653-737f152751c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:32.441620 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c867bdf-d185-4d49-ba4b-5c934cfb3ba4 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:32.463017 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcefaec-2890-4030-8653-737f152751c5 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:32.476895 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e62027c-6678-4151-aa88-d2f85dfa559a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a1319d33e9144859c6010843be312a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:32.477670 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e62027c-6678-4151-aa88-d2f85dfa559a service nova] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:32.480804 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e62027c-6678-4151-aa88-d2f85dfa559a service nova] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:32.481207 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e62027c-6678-4151-aa88-d2f85dfa559a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:32.522211 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e62027c-6678-4151-aa88-d2f85dfa559a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:32.526439 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e62027c-6678-4151-aa88-d2f85dfa559a service nova] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:32.580261 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 851/1714] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:10:32 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:10:32.628272 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c867bdf-d185-4d49-ba4b-5c934cfb3ba4 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:32.644817 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cf67c5-745f-4b11-b95e-fd66778049bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa71c71de263478da234f3c5716a6e5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:32.645459 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cf67c5-745f-4b11-b95e-fd66778049bf service cinder] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:32.645735 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cf67c5-745f-4b11-b95e-fd66778049bf service cinder] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:32.646017 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cf67c5-745f-4b11-b95e-fd66778049bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:32.668114 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cf67c5-745f-4b11-b95e-fd66778049bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:32.669712 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cf67c5-745f-4b11-b95e-fd66778049bf service cinder] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:32.706451 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 864/1715] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:10:32 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:10:32.746555 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e62027c-6678-4151-aa88-d2f85dfa559a service nova] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:32.748190 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-0e62027c-6678-4151-aa88-d2f85dfa559a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:32.886217 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cf67c5-745f-4b11-b95e-fd66778049bf service cinder] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:32.902877 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cf67c5-745f-4b11-b95e-fd66778049bf service cinder] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:32.913982 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1183bc21-8f70-46b7-953a-dd44ba252f1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa71c71de263478da234f3c5716a6e5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:32.919042 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1183bc21-8f70-46b7-953a-dd44ba252f1a service cinder] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:32.919742 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1183bc21-8f70-46b7-953a-dd44ba252f1a service cinder] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:32.920278 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1183bc21-8f70-46b7-953a-dd44ba252f1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:32.941804 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e62027c-6678-4151-aa88-d2f85dfa559a service nova] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:32.951224 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97490f1c-36b6-4349-bad0-771c9461c514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:32.953566 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97490f1c-36b6-4349-bad0-771c9461c514 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:32.953871 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97490f1c-36b6-4349-bad0-771c9461c514 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:32.954145 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97490f1c-36b6-4349-bad0-771c9461c514 admin admin] PATH_INFO: `/v3/users/0a82743949de49f0b32192615c403f84` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:32.970281 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97490f1c-36b6-4349-bad0-771c9461c514 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a82743949de49f0b32192615c403f84)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:32.972750 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97490f1c-36b6-4349-bad0-771c9461c514 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:33.117681 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1183bc21-8f70-46b7-953a-dd44ba252f1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:33.120743 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1183bc21-8f70-46b7-953a-dd44ba252f1a service cinder] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:33.167154 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 865/1716] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:10:32 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 22:10:33.271021 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-97490f1c-36b6-4349-bad0-771c9461c514 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a82743949de49f0b32192615c403f84'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:33.320174 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-97490f1c-36b6-4349-bad0-771c9461c514 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a82743949de49f0b32192615c403f84'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:33.341306 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-97490f1c-36b6-4349-bad0-771c9461c514 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a82743949de49f0b32192615c403f84'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:33.343862 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:10:33.344590 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-97490f1c-36b6-4349-bad0-771c9461c514 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a82743949de49f0b32192615c403f84'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:33.345584 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-97490f1c-36b6-4349-bad0-771c9461c514 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a82743949de49f0b32192615c403f84'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:33.457834 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 852/1717] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:10:32 2022] DELETE /identity/v3/users/0a82743949de49f0b32192615c403f84 => generated 0 bytes in 712 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:10:33.462032 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0293c7-1d71-44e0-b369-f4eed18831eb None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:33.462917 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0293c7-1d71-44e0-b369-f4eed18831eb None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:33.463365 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0293c7-1d71-44e0-b369-f4eed18831eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:33.490826 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-dc0293c7-1d71-44e0-b369-f4eed18831eb None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:33.492485 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-dc0293c7-1d71-44e0-b369-f4eed18831eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:33.619532 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 866/1718] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:10:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:10:33.673709 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97490f1c-36b6-4349-bad0-771c9461c514 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:33.675923 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-97490f1c-36b6-4349-bad0-771c9461c514 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:33.745029 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ff8208-a461-4cbb-8e76-f7a739cf85ad None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:33.745314 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ff8208-a461-4cbb-8e76-f7a739cf85ad None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:33.745585 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ff8208-a461-4cbb-8e76-f7a739cf85ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:33.830586 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-e5ff8208-a461-4cbb-8e76-f7a739cf85ad None None] MFA Rules not processed for user `13f161eb377e4f809b2232c1047c0bb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:33.834440 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-e5ff8208-a461-4cbb-8e76-f7a739cf85ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:33.912891 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97490f1c-36b6-4349-bad0-771c9461c514 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:33.925367 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c7d1e2-1617-4673-a459-1b2154774a61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:33.926001 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c7d1e2-1617-4673-a459-1b2154774a61 service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:33.926279 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c7d1e2-1617-4673-a459-1b2154774a61 service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:33.926548 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c7d1e2-1617-4673-a459-1b2154774a61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:33.954625 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c7d1e2-1617-4673-a459-1b2154774a61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:33.954973 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c7d1e2-1617-4673-a459-1b2154774a61 service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:33.984837 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 853/1719] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:10:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 22:10:33.985601 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c7d1e2-1617-4673-a459-1b2154774a61 service neutron] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:33.987413 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-09c7d1e2-1617-4673-a459-1b2154774a61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:34.031895 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c7d1e2-1617-4673-a459-1b2154774a61 service neutron] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:34.043539 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a579a516-f603-48c8-a2b3-f95467cd5873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:34.044274 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a579a516-f603-48c8-a2b3-f95467cd5873 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:34.044581 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a579a516-f603-48c8-a2b3-f95467cd5873 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:34.044851 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a579a516-f603-48c8-a2b3-f95467cd5873 admin admin] PATH_INFO: `/v3/users/43c7063420214750b53126a323c85e12` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:34.049079 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a579a516-f603-48c8-a2b3-f95467cd5873 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43c7063420214750b53126a323c85e12)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:34.049205 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 867/1720] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:10:33 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 22:10:34.052542 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a579a516-f603-48c8-a2b3-f95467cd5873 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:34.060240 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e57440-9353-44da-8d3f-66d1df3f6e71 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:34.061058 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e57440-9353-44da-8d3f-66d1df3f6e71 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:34.062032 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e57440-9353-44da-8d3f-66d1df3f6e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:34.106395 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-54e57440-9353-44da-8d3f-66d1df3f6e71 None None] MFA Rules not processed for user `13f161eb377e4f809b2232c1047c0bb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:34.112796 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-54e57440-9353-44da-8d3f-66d1df3f6e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:34.142823 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 868/1721] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:10:34 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:10:34.159413 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e57440-9353-44da-8d3f-66d1df3f6e71 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:34.161181 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-54e57440-9353-44da-8d3f-66d1df3f6e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:34.328715 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-a579a516-f603-48c8-a2b3-f95467cd5873 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43c7063420214750b53126a323c85e12'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:34.354561 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e57440-9353-44da-8d3f-66d1df3f6e71 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:34.369352 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247ef74c-f52e-40e1-b189-eea39e5ba66b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a1319d33e9144859c6010843be312a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:34.370143 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247ef74c-f52e-40e1-b189-eea39e5ba66b service nova] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:34.370713 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247ef74c-f52e-40e1-b189-eea39e5ba66b service nova] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:34.371722 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247ef74c-f52e-40e1-b189-eea39e5ba66b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:34.384316 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247ef74c-f52e-40e1-b189-eea39e5ba66b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:34.388444 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247ef74c-f52e-40e1-b189-eea39e5ba66b service nova] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:34.394244 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-a579a516-f603-48c8-a2b3-f95467cd5873 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43c7063420214750b53126a323c85e12'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:34.406145 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 869/1722] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:10:34 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:10:34.422005 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-a579a516-f603-48c8-a2b3-f95467cd5873 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43c7063420214750b53126a323c85e12'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:34.423522 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-a579a516-f603-48c8-a2b3-f95467cd5873 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43c7063420214750b53126a323c85e12'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:34.426952 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-a579a516-f603-48c8-a2b3-f95467cd5873 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43c7063420214750b53126a323c85e12'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:34.533044 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf91695-8d39-48be-9f87-952ee4382a3c None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:34.533434 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf91695-8d39-48be-9f87-952ee4382a3c None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:34.533582 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf91695-8d39-48be-9f87-952ee4382a3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:34.535040 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 854/1723] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:10:33 2022] DELETE /identity/v3/users/43c7063420214750b53126a323c85e12 => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:10:34.585770 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-aaf91695-8d39-48be-9f87-952ee4382a3c None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:34.590341 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-aaf91695-8d39-48be-9f87-952ee4382a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:34.809318 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 870/1724] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:10:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:10:34.825890 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:10:34.847961 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a579a516-f603-48c8-a2b3-f95467cd5873 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:34.863326 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-a579a516-f603-48c8-a2b3-f95467cd5873 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:34.929335 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a579a516-f603-48c8-a2b3-f95467cd5873 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:34.933372 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb01156a-8818-4bbb-99e2-aed4ed97a418 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:34.933998 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb01156a-8818-4bbb-99e2-aed4ed97a418 service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:34.934275 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb01156a-8818-4bbb-99e2-aed4ed97a418 service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:34.934546 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb01156a-8818-4bbb-99e2-aed4ed97a418 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:34.957222 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb01156a-8818-4bbb-99e2-aed4ed97a418 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:34.958845 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb01156a-8818-4bbb-99e2-aed4ed97a418 service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:34.972313 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 855/1725] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:10:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:10:35.484608 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:10:35.551206 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac72a97-c738-4228-9dbb-fb1add23b1c7 None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:35.551567 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac72a97-c738-4228-9dbb-fb1add23b1c7 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:35.551652 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac72a97-c738-4228-9dbb-fb1add23b1c7 None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:35.553167 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 871/1726] 10.222.0.18 () {54 vars in 912 bytes} [Sat Aug 6 22:10:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:10:35.563423 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771b7d5d-8f95-4b2e-80c8-3ccb6a800145 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:35.564056 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771b7d5d-8f95-4b2e-80c8-3ccb6a800145 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:35.564519 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771b7d5d-8f95-4b2e-80c8-3ccb6a800145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:35.695055 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-771b7d5d-8f95-4b2e-80c8-3ccb6a800145 None None] MFA Rules not processed for user `5ceaeadc843c4186b25114b28607fe22`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:35.696629 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-771b7d5d-8f95-4b2e-80c8-3ccb6a800145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:35.705042 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 856/1727] 10.222.0.18 () {58 vars in 1009 bytes} [Sat Aug 6 22:10:35 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 22:10:35.719008 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac72a97-c738-4228-9dbb-fb1add23b1c7 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:35.721413 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-1ac72a97-c738-4228-9dbb-fb1add23b1c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:35.753665 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac72a97-c738-4228-9dbb-fb1add23b1c7 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:35.754353 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771b7d5d-8f95-4b2e-80c8-3ccb6a800145 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:35.758008 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-771b7d5d-8f95-4b2e-80c8-3ccb6a800145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:35.766697 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e35806b-2549-4289-b558-535bad4334e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:35.767518 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e35806b-2549-4289-b558-535bad4334e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:35.767938 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e35806b-2549-4289-b558-535bad4334e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:35.768363 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e35806b-2549-4289-b558-535bad4334e5 admin admin] PATH_INFO: `/v3/projects/b582eb3e54e54444943891b7ea6b3ee4` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:35.770144 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e35806b-2549-4289-b558-535bad4334e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b582eb3e54e54444943891b7ea6b3ee4)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:35.774351 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e35806b-2549-4289-b558-535bad4334e5 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:35.782849 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771b7d5d-8f95-4b2e-80c8-3ccb6a800145 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:35.796665 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4c50dd-a072-43ba-985c-870918898a5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:35.797411 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4c50dd-a072-43ba-985c-870918898a5f service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:35.797683 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4c50dd-a072-43ba-985c-870918898a5f service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:35.798185 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4c50dd-a072-43ba-985c-870918898a5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:35.809595 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4c50dd-a072-43ba-985c-870918898a5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:35.811064 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4c50dd-a072-43ba-985c-870918898a5f service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:35.837106 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 857/1728] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:10:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 22:10:35.933341 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-4e35806b-2549-4289-b558-535bad4334e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b582eb3e54e54444943891b7ea6b3ee4', 'request_id': 'req-4e35806b-2549-4289-b558-535bad4334e5'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:36.048524 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-4e35806b-2549-4289-b558-535bad4334e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b582eb3e54e54444943891b7ea6b3ee4', 'request_id': 'req-4e35806b-2549-4289-b558-535bad4334e5'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:36.066150 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b9ea61-868c-4aa9-af9d-d913cc10176c None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:36.066683 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b9ea61-868c-4aa9-af9d-d913cc10176c None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:36.066683 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b9ea61-868c-4aa9-af9d-d913cc10176c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:36.069123 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-4e35806b-2549-4289-b558-535bad4334e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b582eb3e54e54444943891b7ea6b3ee4', 'request_id': 'req-4e35806b-2549-4289-b558-535bad4334e5'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:36.085826 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-4e35806b-2549-4289-b558-535bad4334e5 admin admin] The token cache is being invalidate because project b582eb3e54e54444943891b7ea6b3ee4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:10:36.086211 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-4e35806b-2549-4289-b558-535bad4334e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:36.088128 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-4e35806b-2549-4289-b558-535bad4334e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:36.092961 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 872/1729] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:10:35 2022] DELETE /identity/v3/projects/b582eb3e54e54444943891b7ea6b3ee4 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:10:36.104367 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-69b9ea61-868c-4aa9-af9d-d913cc10176c None None] MFA Rules not processed for user `0d6e35038fed4b44b986902a5ef55ce8`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:36.105017 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e35806b-2549-4289-b558-535bad4334e5 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:36.106617 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-4e35806b-2549-4289-b558-535bad4334e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:36.106746 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-69b9ea61-868c-4aa9-af9d-d913cc10176c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:36.312026 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 858/1730] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:10:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 22:10:36.315123 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b9ea61-868c-4aa9-af9d-d913cc10176c None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:36.317016 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-69b9ea61-868c-4aa9-af9d-d913cc10176c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:36.359667 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e35806b-2549-4289-b558-535bad4334e5 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:36.372668 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3cd8f8-b6a5-48be-83c6-9921fd167f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:36.373471 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3cd8f8-b6a5-48be-83c6-9921fd167f15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:36.374797 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3cd8f8-b6a5-48be-83c6-9921fd167f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:36.375174 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3cd8f8-b6a5-48be-83c6-9921fd167f15 admin admin] PATH_INFO: `/v3/users/5285d561c8104df5a33452bcfb75c931` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:36.377530 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3cd8f8-b6a5-48be-83c6-9921fd167f15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5285d561c8104df5a33452bcfb75c931)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:36.381364 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3cd8f8-b6a5-48be-83c6-9921fd167f15 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:36.496964 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b9ea61-868c-4aa9-af9d-d913cc10176c None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:36.516266 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80a78c0-d2f3-4cdc-8dc8-00fecef1928e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:36.516971 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80a78c0-d2f3-4cdc-8dc8-00fecef1928e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:36.517293 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80a78c0-d2f3-4cdc-8dc8-00fecef1928e admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:36.517577 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80a78c0-d2f3-4cdc-8dc8-00fecef1928e admin admin] PATH_INFO: `/v3/projects/eac4e8cd9e2c4d0c8a98555d231f049b` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:36.522226 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80a78c0-d2f3-4cdc-8dc8-00fecef1928e admin admin] RBAC: Authorizing `identity:delete_project(project_id=eac4e8cd9e2c4d0c8a98555d231f049b)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:36.524203 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80a78c0-d2f3-4cdc-8dc8-00fecef1928e admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:36.690378 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-7d3cd8f8-b6a5-48be-83c6-9921fd167f15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5285d561c8104df5a33452bcfb75c931'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:36.714656 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-a80a78c0-d2f3-4cdc-8dc8-00fecef1928e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eac4e8cd9e2c4d0c8a98555d231f049b', 'request_id': 'req-a80a78c0-d2f3-4cdc-8dc8-00fecef1928e'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:36.729579 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-7d3cd8f8-b6a5-48be-83c6-9921fd167f15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5285d561c8104df5a33452bcfb75c931'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:36.809063 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-a80a78c0-d2f3-4cdc-8dc8-00fecef1928e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eac4e8cd9e2c4d0c8a98555d231f049b', 'request_id': 'req-a80a78c0-d2f3-4cdc-8dc8-00fecef1928e'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:36.819808 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-a80a78c0-d2f3-4cdc-8dc8-00fecef1928e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eac4e8cd9e2c4d0c8a98555d231f049b', 'request_id': 'req-a80a78c0-d2f3-4cdc-8dc8-00fecef1928e'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:36.843352 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-a80a78c0-d2f3-4cdc-8dc8-00fecef1928e admin admin] The token cache is being invalidate because project eac4e8cd9e2c4d0c8a98555d231f049b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:10:36.848463 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-a80a78c0-d2f3-4cdc-8dc8-00fecef1928e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:36.848463 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-a80a78c0-d2f3-4cdc-8dc8-00fecef1928e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:36.855532 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 859/1731] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:10:36 2022] DELETE /identity/v3/projects/eac4e8cd9e2c4d0c8a98555d231f049b => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:10:36.857158 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1ed05b-185a-423e-a336-f94e25c7c5e6 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:36.857626 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1ed05b-185a-423e-a336-f94e25c7c5e6 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:36.858733 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1ed05b-185a-423e-a336-f94e25c7c5e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:36.895004 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-7d3cd8f8-b6a5-48be-83c6-9921fd167f15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5285d561c8104df5a33452bcfb75c931'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:36.905168 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-7d3cd8f8-b6a5-48be-83c6-9921fd167f15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5285d561c8104df5a33452bcfb75c931'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:36.905168 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-7d3cd8f8-b6a5-48be-83c6-9921fd167f15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5285d561c8104df5a33452bcfb75c931'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:36.936027 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-7f1ed05b-185a-423e-a336-f94e25c7c5e6 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:36.942051 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-7f1ed05b-185a-423e-a336-f94e25c7c5e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:36.959988 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:10:36.965315 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 873/1732] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:10:36 2022] DELETE /identity/v3/users/5285d561c8104df5a33452bcfb75c931 => generated 0 bytes in 861 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:10:36.971908 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cf7f13-990a-471d-8b9f-aae2e2799251 None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:36.977097 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cf7f13-990a-471d-8b9f-aae2e2799251 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:36.979835 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cf7f13-990a-471d-8b9f-aae2e2799251 None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:36.983763 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 874/1733] 10.222.0.18 () {54 vars in 961 bytes} [Sat Aug 6 22:10:36 2022] GET /identity => generated 270 bytes in 18 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:10:36.993756 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b40710-990d-4e55-8ab6-0001323bf774 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:36.994306 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b40710-990d-4e55-8ab6-0001323bf774 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:36.994897 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b40710-990d-4e55-8ab6-0001323bf774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:37.068622 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-b5b40710-990d-4e55-8ab6-0001323bf774 None None] MFA Rules not processed for user `680d62a67ae2456f8c129e26728c3297`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:37.072387 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-b5b40710-990d-4e55-8ab6-0001323bf774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:37.125112 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 860/1734] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:10:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:10:37.130927 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdc3b8e-29d2-4608-b926-e5f65a356560 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:37.134244 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdc3b8e-29d2-4608-b926-e5f65a356560 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:37.134244 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdc3b8e-29d2-4608-b926-e5f65a356560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:37.166753 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-dcdc3b8e-29d2-4608-b926-e5f65a356560 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:37.168201 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-dcdc3b8e-29d2-4608-b926-e5f65a356560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:37.202956 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 875/1735] 10.222.0.18 () {58 vars in 1058 bytes} [Sat Aug 6 22:10:36 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:10:37.212301 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b40710-990d-4e55-8ab6-0001323bf774 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:37.214182 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:10:37.229124 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b40710-990d-4e55-8ab6-0001323bf774 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:37.236068 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d0061a-c4bb-4687-84bc-d5f1c88e1fe7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '680d62a67ae2456f8c129e26728c3297', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:37.236750 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d0061a-c4bb-4687-84bc-d5f1c88e1fe7 service placement] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:37.237035 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d0061a-c4bb-4687-84bc-d5f1c88e1fe7 service placement] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:37.237332 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d0061a-c4bb-4687-84bc-d5f1c88e1fe7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:37.239251 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-d9d0061a-c4bb-4687-84bc-d5f1c88e1fe7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:37.268823 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 861/1736] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:10:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:10:37.269920 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdc3b8e-29d2-4608-b926-e5f65a356560 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:37.283009 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d0061a-c4bb-4687-84bc-d5f1c88e1fe7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:37.286516 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d0061a-c4bb-4687-84bc-d5f1c88e1fe7 service placement] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:37.298504 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdc3b8e-29d2-4608-b926-e5f65a356560 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:37.307936 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e649f498-d706-42ca-86ca-c4f86b4aba39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:37.311376 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e649f498-d706-42ca-86ca-c4f86b4aba39 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:37.311376 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e649f498-d706-42ca-86ca-c4f86b4aba39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:37.311376 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e649f498-d706-42ca-86ca-c4f86b4aba39 admin admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:37.311376 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e649f498-d706-42ca-86ca-c4f86b4aba39 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 22:10:37.316008 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 876/1737] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:10:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 22:10:37.318494 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e649f498-d706-42ca-86ca-c4f86b4aba39 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:37.323556 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e649f498-d706-42ca-86ca-c4f86b4aba39 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:37.349744 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 862/1738] 10.222.0.18 () {58 vars in 1153 bytes} [Sat Aug 6 22:10:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 22:10:37.351839 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d0061a-c4bb-4687-84bc-d5f1c88e1fe7 service placement] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:37.368190 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d0061a-c4bb-4687-84bc-d5f1c88e1fe7 service placement] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:37.378023 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f898c92-2877-4a0a-977a-b79261ff55ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:37.380251 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f898c92-2877-4a0a-977a-b79261ff55ff admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:37.380826 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f898c92-2877-4a0a-977a-b79261ff55ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:37.381608 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f898c92-2877-4a0a-977a-b79261ff55ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:37.386713 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f898c92-2877-4a0a-977a-b79261ff55ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:37.397768 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d33f70-0b6f-4b4f-b88f-c4857eb48792 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:37.398059 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d33f70-0b6f-4b4f-b88f-c4857eb48792 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:37.401869 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d33f70-0b6f-4b4f-b88f-c4857eb48792 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:37.402000 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f898c92-2877-4a0a-977a-b79261ff55ff admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:37.402094 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 863/1739] 10.222.0.18 () {54 vars in 912 bytes} [Sat Aug 6 22:10:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:10:37.409828 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eebea90-aaf4-4a50-a509-3a4ddf93a459 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:37.410138 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eebea90-aaf4-4a50-a509-3a4ddf93a459 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:37.410435 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eebea90-aaf4-4a50-a509-3a4ddf93a459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:37.439772 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-7eebea90-aaf4-4a50-a509-3a4ddf93a459 None None] MFA Rules not processed for user `5ceaeadc843c4186b25114b28607fe22`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:37.441509 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-7eebea90-aaf4-4a50-a509-3a4ddf93a459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:37.541276 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 864/1740] 10.222.0.18 () {58 vars in 1009 bytes} [Sat Aug 6 22:10:37 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 22:10:37.544913 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 877/1741] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:10:37 2022] POST /identity/v3/projects => generated 405 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 22:10:37.546087 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eebea90-aaf4-4a50-a509-3a4ddf93a459 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:37.581529 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f898c92-2877-4a0a-977a-b79261ff55ff admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:37.582996 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eebea90-aaf4-4a50-a509-3a4ddf93a459 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:37.583485 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-4f898c92-2877-4a0a-977a-b79261ff55ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:37.722890 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f898c92-2877-4a0a-977a-b79261ff55ff admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:37.729157 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc62d900-e799-4c22-8170-23db6c06094c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:37.729887 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc62d900-e799-4c22-8170-23db6c06094c service neutron] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:37.730224 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc62d900-e799-4c22-8170-23db6c06094c service neutron] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:37.730646 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc62d900-e799-4c22-8170-23db6c06094c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:37.735176 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e7457d-4efe-4a2c-8772-be496dcfe4bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:37.735800 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e7457d-4efe-4a2c-8772-be496dcfe4bd admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:37.736077 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e7457d-4efe-4a2c-8772-be496dcfe4bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:37.736348 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e7457d-4efe-4a2c-8772-be496dcfe4bd admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:37.737065 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e7457d-4efe-4a2c-8772-be496dcfe4bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:37.739573 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e7457d-4efe-4a2c-8772-be496dcfe4bd admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:37.755417 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc62d900-e799-4c22-8170-23db6c06094c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:37.762409 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc62d900-e799-4c22-8170-23db6c06094c service neutron] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:37.789779 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 878/1742] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:10:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 22:10:37.835868 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 865/1743] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:10:37 2022] POST /identity/v3/users => generated 303 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:10:37.837241 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc62d900-e799-4c22-8170-23db6c06094c service neutron] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:37.868573 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc62d900-e799-4c22-8170-23db6c06094c service neutron] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:37.879894 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e7457d-4efe-4a2c-8772-be496dcfe4bd admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:37.881988 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-71e7457d-4efe-4a2c-8772-be496dcfe4bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:37.889916 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b687284f-063d-4d40-b6a8-90f7e9889dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:37.890984 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b687284f-063d-4d40-b6a8-90f7e9889dc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:37.890984 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b687284f-063d-4d40-b6a8-90f7e9889dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:37.891143 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b687284f-063d-4d40-b6a8-90f7e9889dc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:37.891742 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b687284f-063d-4d40-b6a8-90f7e9889dc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:37.893339 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b687284f-063d-4d40-b6a8-90f7e9889dc0 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:37.913002 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e7457d-4efe-4a2c-8772-be496dcfe4bd admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:37.926589 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff977fdd-c681-4069-ab8f-c8d492c9ef0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:37.926589 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff977fdd-c681-4069-ab8f-c8d492c9ef0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:37.932714 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff977fdd-c681-4069-ab8f-c8d492c9ef0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:37.932714 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff977fdd-c681-4069-ab8f-c8d492c9ef0e admin admin] PATH_INFO: `/v3/projects/50c98ea9dbb345b7a198f03686b52e85` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:37.932852 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 879/1744] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:10:37 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:10:37.932852 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b687284f-063d-4d40-b6a8-90f7e9889dc0 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:37.934205 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff977fdd-c681-4069-ab8f-c8d492c9ef0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=50c98ea9dbb345b7a198f03686b52e85)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:37.936010 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff977fdd-c681-4069-ab8f-c8d492c9ef0e admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:37.971937 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b687284f-063d-4d40-b6a8-90f7e9889dc0 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:38.007752 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b03e552-4402-49c2-a12a-e3bf17817913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:38.008377 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b03e552-4402-49c2-a12a-e3bf17817913 admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:38.008689 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b03e552-4402-49c2-a12a-e3bf17817913 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:38.008961 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b03e552-4402-49c2-a12a-e3bf17817913 admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:38.009623 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b03e552-4402-49c2-a12a-e3bf17817913 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:38.017452 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b03e552-4402-49c2-a12a-e3bf17817913 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:38.040749 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 880/1745] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:10:37 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:10:38.044890 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b03e552-4402-49c2-a12a-e3bf17817913 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:38.067821 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b03e552-4402-49c2-a12a-e3bf17817913 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:38.076307 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d09ea1-8a76-4fd0-a30a-51b9d78279f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:38.078029 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d09ea1-8a76-4fd0-a30a-51b9d78279f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:38.078029 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d09ea1-8a76-4fd0-a30a-51b9d78279f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:38.078202 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d09ea1-8a76-4fd0-a30a-51b9d78279f5 admin admin] PATH_INFO: `/v3/projects/648280526e5944d7a74c866051c0878c/users/10a71cf5bccb4e9b950077a2b9727303/roles/56e4a96094e9434cbbafb22579c034f5` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:38.122431 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d09ea1-8a76-4fd0-a30a-51b9d78279f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=648280526e5944d7a74c866051c0878c, user_id=10a71cf5bccb4e9b950077a2b9727303, role_id=56e4a96094e9434cbbafb22579c034f5)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:38.127447 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d09ea1-8a76-4fd0-a30a-51b9d78279f5 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:38.191332 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:10:38.261221 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 881/1746] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:10:38 2022] PUT /identity/v3/projects/648280526e5944d7a74c866051c0878c/users/10a71cf5bccb4e9b950077a2b9727303/roles/56e4a96094e9434cbbafb22579c034f5 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:10:38.271032 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ddad7-f478-4aa8-bc19-f9af6f51c8b5 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:38.271652 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ddad7-f478-4aa8-bc19-f9af6f51c8b5 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:38.274183 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ddad7-f478-4aa8-bc19-f9af6f51c8b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:38.281291 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-ff977fdd-c681-4069-ab8f-c8d492c9ef0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50c98ea9dbb345b7a198f03686b52e85', 'request_id': 'req-ff977fdd-c681-4069-ab8f-c8d492c9ef0e'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:38.308989 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-157ddad7-f478-4aa8-bc19-f9af6f51c8b5 None None] MFA Rules not processed for user `10a71cf5bccb4e9b950077a2b9727303`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:38.310270 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-157ddad7-f478-4aa8-bc19-f9af6f51c8b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:38.431986 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-ff977fdd-c681-4069-ab8f-c8d492c9ef0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50c98ea9dbb345b7a198f03686b52e85', 'request_id': 'req-ff977fdd-c681-4069-ab8f-c8d492c9ef0e'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:38.439204 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-ff977fdd-c681-4069-ab8f-c8d492c9ef0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50c98ea9dbb345b7a198f03686b52e85', 'request_id': 'req-ff977fdd-c681-4069-ab8f-c8d492c9ef0e'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:38.461108 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 882/1747] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:10:38 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 22:10:38.472513 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157ddad7-f478-4aa8-bc19-f9af6f51c8b5 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:38.475840 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-ff977fdd-c681-4069-ab8f-c8d492c9ef0e admin admin] The token cache is being invalidate because project 50c98ea9dbb345b7a198f03686b52e85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:10:38.477765 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-ff977fdd-c681-4069-ab8f-c8d492c9ef0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:38.482686 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-ff977fdd-c681-4069-ab8f-c8d492c9ef0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:38.485492 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 866/1748] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:10:37 2022] DELETE /identity/v3/projects/50c98ea9dbb345b7a198f03686b52e85 => generated 0 bytes in 606 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:10:38.495163 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f71160-e5a3-4e93-8283-bbdcb5886c39 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:38.495163 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f71160-e5a3-4e93-8283-bbdcb5886c39 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:38.495163 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f71160-e5a3-4e93-8283-bbdcb5886c39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:38.535074 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-42f71160-e5a3-4e93-8283-bbdcb5886c39 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:38.536925 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-42f71160-e5a3-4e93-8283-bbdcb5886c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:38.583423 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157ddad7-f478-4aa8-bc19-f9af6f51c8b5 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:38.588969 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07479595-0ca5-48f1-bd2e-977ec4f50545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:38.589737 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07479595-0ca5-48f1-bd2e-977ec4f50545 admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:38.590092 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07479595-0ca5-48f1-bd2e-977ec4f50545 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:38.590440 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07479595-0ca5-48f1-bd2e-977ec4f50545 admin admin] PATH_INFO: `/v3/projects` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:38.591344 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07479595-0ca5-48f1-bd2e-977ec4f50545 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:38.595168 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07479595-0ca5-48f1-bd2e-977ec4f50545 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:38.613188 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 867/1749] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:10:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 22:10:38.619593 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b62ac3f-5d47-489d-99a2-85b4411d1538 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:38.619593 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b62ac3f-5d47-489d-99a2-85b4411d1538 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:38.619593 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b62ac3f-5d47-489d-99a2-85b4411d1538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:38.636825 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-8b62ac3f-5d47-489d-99a2-85b4411d1538 None None] MFA Rules not processed for user `dfc4de8c76d0470a892dc8bad4d29441`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:38.638035 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-8b62ac3f-5d47-489d-99a2-85b4411d1538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:38.696899 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 868/1750] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:10:38 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:10:38.698008 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea2ba5-a9c3-466a-9069-50ca0a0a1564 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:38.698290 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea2ba5-a9c3-466a-9069-50ca0a0a1564 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:38.698558 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea2ba5-a9c3-466a-9069-50ca0a0a1564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:38.719114 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-a3ea2ba5-a9c3-466a-9069-50ca0a0a1564 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:38.720365 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-a3ea2ba5-a9c3-466a-9069-50ca0a0a1564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:38.732991 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 869/1751] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:10:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:10:38.734745 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ea2ba5-a9c3-466a-9069-50ca0a0a1564 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:38.744571 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ea2ba5-a9c3-466a-9069-50ca0a0a1564 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:38.753688 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:10:38.753953 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 883/1752] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:10:38 2022] POST /identity/v3/projects => generated 403 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:10:38.759473 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07479595-0ca5-48f1-bd2e-977ec4f50545 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:38.760593 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b58d521-f5aa-44ac-9bdd-a06b9387fe7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:38.760699 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b58d521-f5aa-44ac-9bdd-a06b9387fe7d admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:38.760923 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b58d521-f5aa-44ac-9bdd-a06b9387fe7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:38.761195 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b58d521-f5aa-44ac-9bdd-a06b9387fe7d admin admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:38.761873 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b58d521-f5aa-44ac-9bdd-a06b9387fe7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 22:10:38.762173 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b58d521-f5aa-44ac-9bdd-a06b9387fe7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:38.764148 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b58d521-f5aa-44ac-9bdd-a06b9387fe7d admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:38.764914 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-07479595-0ca5-48f1-bd2e-977ec4f50545 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:38.796481 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 870/1753] 10.222.0.18 () {58 vars in 1153 bytes} [Sat Aug 6 22:10:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 22:10:38.797761 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b58d521-f5aa-44ac-9bdd-a06b9387fe7d admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:38.810026 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b58d521-f5aa-44ac-9bdd-a06b9387fe7d admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:38.845639 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07479595-0ca5-48f1-bd2e-977ec4f50545 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:38.850941 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fe1721-33df-4d09-892f-e6ca8b81de95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:38.851817 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fe1721-33df-4d09-892f-e6ca8b81de95 admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:38.852093 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fe1721-33df-4d09-892f-e6ca8b81de95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:38.852360 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fe1721-33df-4d09-892f-e6ca8b81de95 admin admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:38.853022 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fe1721-33df-4d09-892f-e6ca8b81de95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:38.855508 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fe1721-33df-4d09-892f-e6ca8b81de95 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:38.891820 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a8bc4f-0f9c-4eb8-a1f9-07d26222f9d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:38.892463 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a8bc4f-0f9c-4eb8-a1f9-07d26222f9d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:38.892749 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a8bc4f-0f9c-4eb8-a1f9-07d26222f9d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:38.893033 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a8bc4f-0f9c-4eb8-a1f9-07d26222f9d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:38.893654 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a8bc4f-0f9c-4eb8-a1f9-07d26222f9d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:38.895584 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a8bc4f-0f9c-4eb8-a1f9-07d26222f9d8 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:39.015379 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 871/1754] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:10:38 2022] POST /identity/v3/projects => generated 377 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:10:39.016505 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a8bc4f-0f9c-4eb8-a1f9-07d26222f9d8 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:39.097533 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 884/1755] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:10:38 2022] POST /identity/v3/users => generated 308 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:10:39.098911 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fe1721-33df-4d09-892f-e6ca8b81de95 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:39.130925 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fe1721-33df-4d09-892f-e6ca8b81de95 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:39.185218 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a8bc4f-0f9c-4eb8-a1f9-07d26222f9d8 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:39.204491 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08bf64a-d90e-4f3d-a3ec-03e8c05159cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:39.205118 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08bf64a-d90e-4f3d-a3ec-03e8c05159cb admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:39.205388 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08bf64a-d90e-4f3d-a3ec-03e8c05159cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:39.205691 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08bf64a-d90e-4f3d-a3ec-03e8c05159cb admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:39.206282 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08bf64a-d90e-4f3d-a3ec-03e8c05159cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:39.208199 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08bf64a-d90e-4f3d-a3ec-03e8c05159cb admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:39.323537 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7db6dc9-2f16-4c19-9633-f593af51f596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:39.324810 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7db6dc9-2f16-4c19-9633-f593af51f596 admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:39.325065 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7db6dc9-2f16-4c19-9633-f593af51f596 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:39.327130 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7db6dc9-2f16-4c19-9633-f593af51f596 admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:39.327939 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7db6dc9-2f16-4c19-9633-f593af51f596 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:39.335636 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7db6dc9-2f16-4c19-9633-f593af51f596 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:39.340564 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 872/1756] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:10:39 2022] POST /identity/v3/users => generated 289 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 22:10:39.342811 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08bf64a-d90e-4f3d-a3ec-03e8c05159cb admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:39.361263 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 885/1757] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:10:39 2022] GET /identity/v3/roles => generated 1420 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:10:39.365136 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7db6dc9-2f16-4c19-9633-f593af51f596 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:39.366565 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08bf64a-d90e-4f3d-a3ec-03e8c05159cb admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:39.380381 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acc2c43-308c-4cd7-9c9e-c34e05d9f068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:39.381128 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acc2c43-308c-4cd7-9c9e-c34e05d9f068 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:39.381519 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acc2c43-308c-4cd7-9c9e-c34e05d9f068 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:39.382150 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acc2c43-308c-4cd7-9c9e-c34e05d9f068 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:39.389783 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acc2c43-308c-4cd7-9c9e-c34e05d9f068 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:39.389783 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acc2c43-308c-4cd7-9c9e-c34e05d9f068 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:39.391139 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7db6dc9-2f16-4c19-9633-f593af51f596 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:39.403330 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9ecef9-3da9-4f77-9794-9c68a7e3b3b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:39.404089 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9ecef9-3da9-4f77-9794-9c68a7e3b3b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:39.404455 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9ecef9-3da9-4f77-9794-9c68a7e3b3b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:39.405614 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9ecef9-3da9-4f77-9794-9c68a7e3b3b4 admin admin] PATH_INFO: `/v3/projects/3c8e0095e0e740eca2264cceb5169bb1/users/895d8283f69b4004940cb2c3a9a37ab2/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:39.411349 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 873/1758] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:10:39 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:10:39.413444 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acc2c43-308c-4cd7-9c9e-c34e05d9f068 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:39.431338 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acc2c43-308c-4cd7-9c9e-c34e05d9f068 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:39.437022 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7ad1a0-722c-4bba-8676-0ad43f21ea4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:39.437882 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ad1a0-722c-4bba-8676-0ad43f21ea4e admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:39.438263 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ad1a0-722c-4bba-8676-0ad43f21ea4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:39.438622 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ad1a0-722c-4bba-8676-0ad43f21ea4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:39.439323 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7ad1a0-722c-4bba-8676-0ad43f21ea4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:39.440838 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9ecef9-3da9-4f77-9794-9c68a7e3b3b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c8e0095e0e740eca2264cceb5169bb1, user_id=895d8283f69b4004940cb2c3a9a37ab2, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:39.441165 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7ad1a0-722c-4bba-8676-0ad43f21ea4e admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:39.443636 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9ecef9-3da9-4f77-9794-9c68a7e3b3b4 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:39.470389 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 874/1759] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:10:39 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:10:39.474616 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7ad1a0-722c-4bba-8676-0ad43f21ea4e admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:39.487451 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7ad1a0-722c-4bba-8676-0ad43f21ea4e admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:39.495553 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e83e4c7-8cfa-4264-85c8-e75eb7c2ac86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:39.498029 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e83e4c7-8cfa-4264-85c8-e75eb7c2ac86 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:39.498146 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e83e4c7-8cfa-4264-85c8-e75eb7c2ac86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:39.500329 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e83e4c7-8cfa-4264-85c8-e75eb7c2ac86 admin admin] PATH_INFO: `/v3/projects/3ae1ea2ce54d47c4829ad2ad39d1eb77/users/d20a99eab400474c88f3e42754d14080/roles/56e4a96094e9434cbbafb22579c034f5` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:39.526317 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e83e4c7-8cfa-4264-85c8-e75eb7c2ac86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ae1ea2ce54d47c4829ad2ad39d1eb77, user_id=d20a99eab400474c88f3e42754d14080, role_id=56e4a96094e9434cbbafb22579c034f5)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:39.531450 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e83e4c7-8cfa-4264-85c8-e75eb7c2ac86 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:39.557749 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 886/1760] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:10:39 2022] PUT /identity/v3/projects/3c8e0095e0e740eca2264cceb5169bb1/users/895d8283f69b4004940cb2c3a9a37ab2/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:10:39.562313 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9ecef9-3da9-4f77-9794-9c68a7e3b3b4 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:39.660488 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 875/1761] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:10:39 2022] PUT /identity/v3/projects/3ae1ea2ce54d47c4829ad2ad39d1eb77/users/d20a99eab400474c88f3e42754d14080/roles/56e4a96094e9434cbbafb22579c034f5 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 22:10:39.675387 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d5e76e-962b-4bb3-a679-dcb2b5e53e46 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:39.676052 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d5e76e-962b-4bb3-a679-dcb2b5e53e46 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:39.676673 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d5e76e-962b-4bb3-a679-dcb2b5e53e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:39.703795 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9ecef9-3da9-4f77-9794-9c68a7e3b3b4 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:39.738812 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-d3d5e76e-962b-4bb3-a679-dcb2b5e53e46 None None] MFA Rules not processed for user `d20a99eab400474c88f3e42754d14080`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:39.740040 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-d3d5e76e-962b-4bb3-a679-dcb2b5e53e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:39.777860 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775c6ab4-40d0-4fc1-9d32-32bcb4329830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:39.778857 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775c6ab4-40d0-4fc1-9d32-32bcb4329830 admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:39.778857 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775c6ab4-40d0-4fc1-9d32-32bcb4329830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:39.779109 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775c6ab4-40d0-4fc1-9d32-32bcb4329830 admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:39.779843 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775c6ab4-40d0-4fc1-9d32-32bcb4329830 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:39.781819 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775c6ab4-40d0-4fc1-9d32-32bcb4329830 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:39.810542 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 887/1762] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:10:39 2022] GET /identity/v3/roles => generated 1420 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:10:39.817772 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775c6ab4-40d0-4fc1-9d32-32bcb4329830 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:39.830997 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775c6ab4-40d0-4fc1-9d32-32bcb4329830 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:39.832221 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 876/1763] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:10:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:10:39.833220 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d5e76e-962b-4bb3-a679-dcb2b5e53e46 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:39.839452 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e52d7c-a196-4d71-8ead-297be9661bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:39.849710 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e52d7c-a196-4d71-8ead-297be9661bcc admin admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:39.849710 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e52d7c-a196-4d71-8ead-297be9661bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:39.849710 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e52d7c-a196-4d71-8ead-297be9661bcc admin admin] PATH_INFO: `/v3/projects/3c8e0095e0e740eca2264cceb5169bb1/users/895d8283f69b4004940cb2c3a9a37ab2/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:39.849938 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d5e76e-962b-4bb3-a679-dcb2b5e53e46 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:39.849938 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de31175-325f-401c-a2b1-4f0a5f1fb1fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:39.850099 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de31175-325f-401c-a2b1-4f0a5f1fb1fe admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:39.850327 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de31175-325f-401c-a2b1-4f0a5f1fb1fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:39.850605 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de31175-325f-401c-a2b1-4f0a5f1fb1fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:39.853741 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de31175-325f-401c-a2b1-4f0a5f1fb1fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:39.854465 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de31175-325f-401c-a2b1-4f0a5f1fb1fe admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:39.855423 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e52d7c-a196-4d71-8ead-297be9661bcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c8e0095e0e740eca2264cceb5169bb1, user_id=895d8283f69b4004940cb2c3a9a37ab2, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:39.859806 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e52d7c-a196-4d71-8ead-297be9661bcc admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:39.874662 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 888/1764] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:10:39 2022] PUT /identity/v3/projects/3c8e0095e0e740eca2264cceb5169bb1/users/895d8283f69b4004940cb2c3a9a37ab2/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:10:39.885770 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc484193-b526-4d4c-b64f-389f82395937 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:39.885983 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc484193-b526-4d4c-b64f-389f82395937 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:39.886260 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc484193-b526-4d4c-b64f-389f82395937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:39.935058 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-dc484193-b526-4d4c-b64f-389f82395937 None None] MFA Rules not processed for user `895d8283f69b4004940cb2c3a9a37ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:39.937807 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-dc484193-b526-4d4c-b64f-389f82395937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:40.006815 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 877/1765] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:10:39 2022] POST /identity/v3/projects => generated 377 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:10:40.013098 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de31175-325f-401c-a2b1-4f0a5f1fb1fe admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:40.079812 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 889/1766] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:10:39 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:10:40.087931 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc484193-b526-4d4c-b64f-389f82395937 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:40.089619 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-dc484193-b526-4d4c-b64f-389f82395937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:40.105626 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:10:40.131563 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de31175-325f-401c-a2b1-4f0a5f1fb1fe admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:40.137175 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1149c606-ed41-482f-9a09-63d5b9bd6739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:40.137762 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1149c606-ed41-482f-9a09-63d5b9bd6739 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:40.138045 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1149c606-ed41-482f-9a09-63d5b9bd6739 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:40.138325 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1149c606-ed41-482f-9a09-63d5b9bd6739 admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:40.138962 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1149c606-ed41-482f-9a09-63d5b9bd6739 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:40.141489 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1149c606-ed41-482f-9a09-63d5b9bd6739 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:40.164484 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc484193-b526-4d4c-b64f-389f82395937 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:40.219523 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779b4e6a-cb4f-446b-8cb8-a7822d21a513 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa71c71de263478da234f3c5716a6e5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:40.221683 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b4e6a-cb4f-446b-8cb8-a7822d21a513 service cinder] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:40.221683 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b4e6a-cb4f-446b-8cb8-a7822d21a513 service cinder] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:40.221683 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b4e6a-cb4f-446b-8cb8-a7822d21a513 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:40.288633 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 878/1767] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:10:40 2022] POST /identity/v3/users => generated 295 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:10:40.292442 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1149c606-ed41-482f-9a09-63d5b9bd6739 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:40.303664 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1149c606-ed41-482f-9a09-63d5b9bd6739 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:40.310505 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2d519c-1252-401e-a010-b6d768d83841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:40.312156 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2d519c-1252-401e-a010-b6d768d83841 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:40.312156 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2d519c-1252-401e-a010-b6d768d83841 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:40.312612 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2d519c-1252-401e-a010-b6d768d83841 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:40.313984 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2d519c-1252-401e-a010-b6d768d83841 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:40.316358 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2d519c-1252-401e-a010-b6d768d83841 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:40.324719 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779b4e6a-cb4f-446b-8cb8-a7822d21a513 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:40.338478 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 879/1768] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:10:40 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:10:40.338602 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779b4e6a-cb4f-446b-8cb8-a7822d21a513 service cinder] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:40.339561 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2d519c-1252-401e-a010-b6d768d83841 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:40.375440 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 890/1769] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:10:40 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:10:40.375812 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2d519c-1252-401e-a010-b6d768d83841 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:40.406411 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ec3edd-7f20-4621-942d-9c98c0720d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:40.407620 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ec3edd-7f20-4621-942d-9c98c0720d6b admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:40.408006 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ec3edd-7f20-4621-942d-9c98c0720d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:40.408352 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ec3edd-7f20-4621-942d-9c98c0720d6b admin admin] PATH_INFO: `/v3/projects/a796972f745449b68ae0855452099f1b/users/6eda50a5b7a14ccc989be8c46b992990/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:40.465887 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ec3edd-7f20-4621-942d-9c98c0720d6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a796972f745449b68ae0855452099f1b, user_id=6eda50a5b7a14ccc989be8c46b992990, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:40.468862 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ec3edd-7f20-4621-942d-9c98c0720d6b admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:40.556214 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 880/1770] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:10:40 2022] PUT /identity/v3/projects/a796972f745449b68ae0855452099f1b/users/6eda50a5b7a14ccc989be8c46b992990/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 22:10:40.557016 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779b4e6a-cb4f-446b-8cb8-a7822d21a513 service cinder] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:40.570912 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779b4e6a-cb4f-446b-8cb8-a7822d21a513 service cinder] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:40.661220 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c62b8a-d08c-4984-9681-9d949c6d61b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:40.661910 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c62b8a-d08c-4984-9681-9d949c6d61b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:40.662198 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c62b8a-d08c-4984-9681-9d949c6d61b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:40.662617 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c62b8a-d08c-4984-9681-9d949c6d61b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:40.663358 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c62b8a-d08c-4984-9681-9d949c6d61b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:40.665149 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c62b8a-d08c-4984-9681-9d949c6d61b4 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:40.676508 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 891/1771] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:10:40 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:10:40.680768 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ec3edd-7f20-4621-942d-9c98c0720d6b admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:40.692018 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ec3edd-7f20-4621-942d-9c98c0720d6b admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:40.696957 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91097791-57d3-4b46-a8eb-1e9ddf93d081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:40.697834 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91097791-57d3-4b46-a8eb-1e9ddf93d081 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:40.698292 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91097791-57d3-4b46-a8eb-1e9ddf93d081 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:40.698857 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91097791-57d3-4b46-a8eb-1e9ddf93d081 admin admin] PATH_INFO: `/v3/projects/a796972f745449b68ae0855452099f1b/users/6eda50a5b7a14ccc989be8c46b992990/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:40.701041 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91097791-57d3-4b46-a8eb-1e9ddf93d081 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a796972f745449b68ae0855452099f1b, user_id=6eda50a5b7a14ccc989be8c46b992990, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:40.704222 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91097791-57d3-4b46-a8eb-1e9ddf93d081 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:40.715395 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 881/1772] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:10:40 2022] PUT /identity/v3/projects/a796972f745449b68ae0855452099f1b/users/6eda50a5b7a14ccc989be8c46b992990/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:10:40.725854 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92d7475-0fb0-4b18-a93e-89f91fdb5f8f None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:40.726508 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92d7475-0fb0-4b18-a93e-89f91fdb5f8f None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:40.726848 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92d7475-0fb0-4b18-a93e-89f91fdb5f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:40.759187 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-e92d7475-0fb0-4b18-a93e-89f91fdb5f8f None None] MFA Rules not processed for user `6eda50a5b7a14ccc989be8c46b992990`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:40.760625 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-e92d7475-0fb0-4b18-a93e-89f91fdb5f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:40.847547 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 892/1773] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:10:40 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:10:40.859597 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91097791-57d3-4b46-a8eb-1e9ddf93d081 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:40.861340 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-91097791-57d3-4b46-a8eb-1e9ddf93d081 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:40.926079 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91097791-57d3-4b46-a8eb-1e9ddf93d081 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:40.929645 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7cfc0b-f765-4d6b-aa4c-15aef73a904a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa71c71de263478da234f3c5716a6e5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:40.930274 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7cfc0b-f765-4d6b-aa4c-15aef73a904a service cinder] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:40.930556 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7cfc0b-f765-4d6b-aa4c-15aef73a904a service cinder] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:40.930882 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7cfc0b-f765-4d6b-aa4c-15aef73a904a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:41.002225 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7cfc0b-f765-4d6b-aa4c-15aef73a904a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:41.003687 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7cfc0b-f765-4d6b-aa4c-15aef73a904a service cinder] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:41.020694 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 882/1774] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:10:40 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 22:10:41.054234 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:10:41.590616 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:10:41.756661 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:10:42.842622 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:10:42.951045 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:10:43.522497 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:10:43.585130 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:10:43.848205 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92d7475-0fb0-4b18-a93e-89f91fdb5f8f None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:43.917561 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92d7475-0fb0-4b18-a93e-89f91fdb5f8f None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:43.924948 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed7b288-1e14-46d6-bade-18419c16f500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:43.925600 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed7b288-1e14-46d6-bade-18419c16f500 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:43.925921 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed7b288-1e14-46d6-bade-18419c16f500 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:43.926201 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed7b288-1e14-46d6-bade-18419c16f500 admin admin] PATH_INFO: `/v3/users/10a71cf5bccb4e9b950077a2b9727303` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:43.927614 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed7b288-1e14-46d6-bade-18419c16f500 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10a71cf5bccb4e9b950077a2b9727303)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:43.929822 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed7b288-1e14-46d6-bade-18419c16f500 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:44.173881 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:10:44.222966 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-eed7b288-1e14-46d6-bade-18419c16f500 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10a71cf5bccb4e9b950077a2b9727303'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:44.236756 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-eed7b288-1e14-46d6-bade-18419c16f500 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10a71cf5bccb4e9b950077a2b9727303'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:44.356828 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-eed7b288-1e14-46d6-bade-18419c16f500 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10a71cf5bccb4e9b950077a2b9727303'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:44.367039 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-eed7b288-1e14-46d6-bade-18419c16f500 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10a71cf5bccb4e9b950077a2b9727303'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:44.367922 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-eed7b288-1e14-46d6-bade-18419c16f500 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10a71cf5bccb4e9b950077a2b9727303'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:44.377654 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 893/1775] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:10:43 2022] DELETE /identity/v3/users/10a71cf5bccb4e9b950077a2b9727303 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:10:44.380292 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8784e6c2-5e76-4d35-881a-b37398175fe5 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:44.380627 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8784e6c2-5e76-4d35-881a-b37398175fe5 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:44.380903 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8784e6c2-5e76-4d35-881a-b37398175fe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:44.408244 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-8784e6c2-5e76-4d35-881a-b37398175fe5 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:44.409502 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-8784e6c2-5e76-4d35-881a-b37398175fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:44.488480 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 883/1776] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:10:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:10:44.498943 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed7b288-1e14-46d6-bade-18419c16f500 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:44.500830 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-eed7b288-1e14-46d6-bade-18419c16f500 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:44.561793 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed7b288-1e14-46d6-bade-18419c16f500 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:44.565372 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc16234-5ab8-4160-b224-b13a599e6078 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:44.565985 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc16234-5ab8-4160-b224-b13a599e6078 service neutron] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:44.566268 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc16234-5ab8-4160-b224-b13a599e6078 service neutron] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:44.566572 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc16234-5ab8-4160-b224-b13a599e6078 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:44.575885 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc16234-5ab8-4160-b224-b13a599e6078 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:44.577744 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc16234-5ab8-4160-b224-b13a599e6078 service neutron] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:44.587534 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 894/1777] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:10:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:10:44.710072 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:10:45.181057 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8784e6c2-5e76-4d35-881a-b37398175fe5 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:45.186995 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-8784e6c2-5e76-4d35-881a-b37398175fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:45.204454 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8784e6c2-5e76-4d35-881a-b37398175fe5 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:45.214986 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2003a8bf-7b60-4148-a8fa-861e0d8a08d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:45.215910 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2003a8bf-7b60-4148-a8fa-861e0d8a08d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:45.216325 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2003a8bf-7b60-4148-a8fa-861e0d8a08d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:45.216760 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2003a8bf-7b60-4148-a8fa-861e0d8a08d6 admin admin] PATH_INFO: `/v3/projects/648280526e5944d7a74c866051c0878c` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:45.218735 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2003a8bf-7b60-4148-a8fa-861e0d8a08d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=648280526e5944d7a74c866051c0878c)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:45.221717 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2003a8bf-7b60-4148-a8fa-861e0d8a08d6 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:45.444333 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-2003a8bf-7b60-4148-a8fa-861e0d8a08d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '648280526e5944d7a74c866051c0878c', 'request_id': 'req-2003a8bf-7b60-4148-a8fa-861e0d8a08d6'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:45.568083 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-2003a8bf-7b60-4148-a8fa-861e0d8a08d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '648280526e5944d7a74c866051c0878c', 'request_id': 'req-2003a8bf-7b60-4148-a8fa-861e0d8a08d6'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:45.577181 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-2003a8bf-7b60-4148-a8fa-861e0d8a08d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '648280526e5944d7a74c866051c0878c', 'request_id': 'req-2003a8bf-7b60-4148-a8fa-861e0d8a08d6'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:45.592655 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-2003a8bf-7b60-4148-a8fa-861e0d8a08d6 admin admin] The token cache is being invalidate because project 648280526e5944d7a74c866051c0878c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:10:45.593160 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-2003a8bf-7b60-4148-a8fa-861e0d8a08d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:45.593986 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-2003a8bf-7b60-4148-a8fa-861e0d8a08d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:45.596181 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 884/1778] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:10:45 2022] DELETE /identity/v3/projects/648280526e5944d7a74c866051c0878c => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:10:45.604995 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc16234-5ab8-4160-b224-b13a599e6078 service neutron] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:45.616566 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc16234-5ab8-4160-b224-b13a599e6078 service neutron] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:45.694848 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513edb6b-6e90-4872-886e-ff6c24801a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:45.695795 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513edb6b-6e90-4872-886e-ff6c24801a2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:45.696220 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513edb6b-6e90-4872-886e-ff6c24801a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:45.696693 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513edb6b-6e90-4872-886e-ff6c24801a2c admin admin] PATH_INFO: `/v3/users/895d8283f69b4004940cb2c3a9a37ab2` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:45.699211 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513edb6b-6e90-4872-886e-ff6c24801a2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=895d8283f69b4004940cb2c3a9a37ab2)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:45.702178 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513edb6b-6e90-4872-886e-ff6c24801a2c admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:46.145141 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-513edb6b-6e90-4872-886e-ff6c24801a2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '895d8283f69b4004940cb2c3a9a37ab2'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:46.160192 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-513edb6b-6e90-4872-886e-ff6c24801a2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '895d8283f69b4004940cb2c3a9a37ab2'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:46.196435 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:10:46.282407 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-513edb6b-6e90-4872-886e-ff6c24801a2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '895d8283f69b4004940cb2c3a9a37ab2'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:46.282732 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:10:46.293283 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-513edb6b-6e90-4872-886e-ff6c24801a2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '895d8283f69b4004940cb2c3a9a37ab2'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:46.294226 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-513edb6b-6e90-4872-886e-ff6c24801a2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '895d8283f69b4004940cb2c3a9a37ab2'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:46.333752 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 895/1779] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:10:45 2022] DELETE /identity/v3/users/895d8283f69b4004940cb2c3a9a37ab2 => generated 0 bytes in 729 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:10:46.591293 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fd56ae-df8a-4253-aeb8-664c314d049b None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:46.591571 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fd56ae-df8a-4253-aeb8-664c314d049b None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:46.592548 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fd56ae-df8a-4253-aeb8-664c314d049b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:46.636642 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:10:46.682283 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-b6fd56ae-df8a-4253-aeb8-664c314d049b None None] MFA Rules not processed for user `d1b26ba592724dd389584bb800d877a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:46.683958 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-b6fd56ae-df8a-4253-aeb8-664c314d049b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:46.769079 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 885/1780] 10.222.0.18 () {58 vars in 1009 bytes} [Sat Aug 6 22:10:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:10:47.062622 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513edb6b-6e90-4872-886e-ff6c24801a2c admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:47.064218 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-513edb6b-6e90-4872-886e-ff6c24801a2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:47.141550 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513edb6b-6e90-4872-886e-ff6c24801a2c admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:47.156865 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e010d478-63b6-47d2-a284-656a69637876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:47.157153 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e010d478-63b6-47d2-a284-656a69637876 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:47.157438 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e010d478-63b6-47d2-a284-656a69637876 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:47.158901 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e010d478-63b6-47d2-a284-656a69637876 admin admin] PATH_INFO: `/v3/users/3c37a7f2f79e4a4183a626fe6f6b5834` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:47.165511 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e010d478-63b6-47d2-a284-656a69637876 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c37a7f2f79e4a4183a626fe6f6b5834)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:47.177033 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e010d478-63b6-47d2-a284-656a69637876 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:47.462121 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fd56ae-df8a-4253-aeb8-664c314d049b None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:47.464735 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-b6fd56ae-df8a-4253-aeb8-664c314d049b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:47.548551 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-e010d478-63b6-47d2-a284-656a69637876 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c37a7f2f79e4a4183a626fe6f6b5834'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:47.572109 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-e010d478-63b6-47d2-a284-656a69637876 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c37a7f2f79e4a4183a626fe6f6b5834'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:47.648079 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fd56ae-df8a-4253-aeb8-664c314d049b None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:47.656796 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af3fdc4-a9d0-4c8b-94f4-97ff32c8c7e4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7fc9da7505744608dfd3bbd46a0a9d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:47.657464 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af3fdc4-a9d0-4c8b-94f4-97ff32c8c7e4 service swift] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:47.657752 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af3fdc4-a9d0-4c8b-94f4-97ff32c8c7e4 service swift] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:47.658269 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af3fdc4-a9d0-4c8b-94f4-97ff32c8c7e4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:47.675464 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af3fdc4-a9d0-4c8b-94f4-97ff32c8c7e4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:47.677605 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af3fdc4-a9d0-4c8b-94f4-97ff32c8c7e4 service swift] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:47.704396 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 886/1781] 10.222.0.18 () {60 vars in 1384 bytes} [Sat Aug 6 22:10:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 06 22:10:47.706080 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23749174-704f-4fdb-9a66-ea45b497806e None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:47.706446 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23749174-704f-4fdb-9a66-ea45b497806e None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:47.706801 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23749174-704f-4fdb-9a66-ea45b497806e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:47.721362 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-e010d478-63b6-47d2-a284-656a69637876 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c37a7f2f79e4a4183a626fe6f6b5834'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:47.731503 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-e010d478-63b6-47d2-a284-656a69637876 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c37a7f2f79e4a4183a626fe6f6b5834'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:47.738352 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-e010d478-63b6-47d2-a284-656a69637876 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c37a7f2f79e4a4183a626fe6f6b5834'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:47.756611 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-23749174-704f-4fdb-9a66-ea45b497806e None None] MFA Rules not processed for user `5ceaeadc843c4186b25114b28607fe22`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:47.758074 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-23749174-704f-4fdb-9a66-ea45b497806e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:47.761731 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 896/1782] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:10:47 2022] DELETE /identity/v3/users/3c37a7f2f79e4a4183a626fe6f6b5834 => generated 0 bytes in 699 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:10:47.777663 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a9247d-80ee-48e8-bdb4-34147c6d6b92 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:47.781639 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a9247d-80ee-48e8-bdb4-34147c6d6b92 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:47.784349 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a9247d-80ee-48e8-bdb4-34147c6d6b92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:47.844287 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-77a9247d-80ee-48e8-bdb4-34147c6d6b92 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:47.849711 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-77a9247d-80ee-48e8-bdb4-34147c6d6b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:47.871612 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 887/1783] 10.222.0.43 () {58 vars in 1015 bytes} [Sat Aug 6 22:10:47 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:10:47.906343 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:10:48.015529 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 897/1784] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:10:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:10:48.023596 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23749174-704f-4fdb-9a66-ea45b497806e None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:48.025222 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-23749174-704f-4fdb-9a66-ea45b497806e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:48.039617 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23749174-704f-4fdb-9a66-ea45b497806e None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:48.044721 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeda4b08-80ba-4b78-9b77-d9048f2b27f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:48.047965 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeda4b08-80ba-4b78-9b77-d9048f2b27f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:48.047965 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeda4b08-80ba-4b78-9b77-d9048f2b27f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:48.047965 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeda4b08-80ba-4b78-9b77-d9048f2b27f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:48.064082 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a9247d-80ee-48e8-bdb4-34147c6d6b92 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:48.064310 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-77a9247d-80ee-48e8-bdb4-34147c6d6b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:48.078791 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a9247d-80ee-48e8-bdb4-34147c6d6b92 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:48.078931 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeda4b08-80ba-4b78-9b77-d9048f2b27f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:48.081308 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeda4b08-80ba-4b78-9b77-d9048f2b27f8 service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:48.101487 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0a2c5e-4d4e-4b23-aa68-c32e3303e4e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:48.102156 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0a2c5e-4d4e-4b23-aa68-c32e3303e4e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:48.102436 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0a2c5e-4d4e-4b23-aa68-c32e3303e4e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:48.102731 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0a2c5e-4d4e-4b23-aa68-c32e3303e4e4 admin admin] PATH_INFO: `/v3/projects/3c8e0095e0e740eca2264cceb5169bb1` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:48.103938 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0a2c5e-4d4e-4b23-aa68-c32e3303e4e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c8e0095e0e740eca2264cceb5169bb1)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:48.105916 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0a2c5e-4d4e-4b23-aa68-c32e3303e4e4 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:48.112560 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 888/1785] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:10:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 22:10:48.283759 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-bb0a2c5e-4d4e-4b23-aa68-c32e3303e4e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c8e0095e0e740eca2264cceb5169bb1', 'request_id': 'req-bb0a2c5e-4d4e-4b23-aa68-c32e3303e4e4'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:48.362625 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-bb0a2c5e-4d4e-4b23-aa68-c32e3303e4e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c8e0095e0e740eca2264cceb5169bb1', 'request_id': 'req-bb0a2c5e-4d4e-4b23-aa68-c32e3303e4e4'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:48.370275 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-bb0a2c5e-4d4e-4b23-aa68-c32e3303e4e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c8e0095e0e740eca2264cceb5169bb1', 'request_id': 'req-bb0a2c5e-4d4e-4b23-aa68-c32e3303e4e4'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:48.400250 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-bb0a2c5e-4d4e-4b23-aa68-c32e3303e4e4 admin admin] The token cache is being invalidate because project 3c8e0095e0e740eca2264cceb5169bb1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:10:48.400250 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-bb0a2c5e-4d4e-4b23-aa68-c32e3303e4e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:48.400250 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-bb0a2c5e-4d4e-4b23-aa68-c32e3303e4e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:48.401791 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 898/1786] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:10:48 2022] DELETE /identity/v3/projects/3c8e0095e0e740eca2264cceb5169bb1 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 22:10:48.407847 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae916134-fa29-44be-9c1d-31f439bd6ceb None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:48.408497 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae916134-fa29-44be-9c1d-31f439bd6ceb None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:48.408954 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae916134-fa29-44be-9c1d-31f439bd6ceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:48.438740 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-ae916134-fa29-44be-9c1d-31f439bd6ceb None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:48.440136 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-ae916134-fa29-44be-9c1d-31f439bd6ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:48.527819 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:10:48.596529 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 889/1787] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:10:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:10:48.597980 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61433a00-b70d-4ece-a21e-a0546257f325 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:48.598376 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61433a00-b70d-4ece-a21e-a0546257f325 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:48.598664 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61433a00-b70d-4ece-a21e-a0546257f325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:48.627518 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-61433a00-b70d-4ece-a21e-a0546257f325 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:48.629546 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-61433a00-b70d-4ece-a21e-a0546257f325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:48.648688 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae916134-fa29-44be-9c1d-31f439bd6ceb None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:48.649023 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 899/1788] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:10:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:10:48.662089 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae916134-fa29-44be-9c1d-31f439bd6ceb None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:48.686858 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d47827-58ec-4267-bb79-0eace56e0137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:48.689262 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d47827-58ec-4267-bb79-0eace56e0137 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:48.689726 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d47827-58ec-4267-bb79-0eace56e0137 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:48.690008 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d47827-58ec-4267-bb79-0eace56e0137 admin admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:48.690918 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d47827-58ec-4267-bb79-0eace56e0137 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 22:10:48.692860 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d47827-58ec-4267-bb79-0eace56e0137 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:48.701230 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d47827-58ec-4267-bb79-0eace56e0137 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:48.929705 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 890/1789] 10.222.0.18 () {58 vars in 1153 bytes} [Sat Aug 6 22:10:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 22:10:48.930029 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61433a00-b70d-4ece-a21e-a0546257f325 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:48.938824 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61433a00-b70d-4ece-a21e-a0546257f325 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:48.955513 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dfef76-e0a0-4dc2-85a5-f4dd2d453a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:48.974972 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dfef76-e0a0-4dc2-85a5-f4dd2d453a74 admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:48.974972 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dfef76-e0a0-4dc2-85a5-f4dd2d453a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:48.974972 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dfef76-e0a0-4dc2-85a5-f4dd2d453a74 admin admin] PATH_INFO: `/v3/projects` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:48.974972 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dfef76-e0a0-4dc2-85a5-f4dd2d453a74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:48.974972 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dfef76-e0a0-4dc2-85a5-f4dd2d453a74 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:49.319603 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 900/1790] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:10:48 2022] POST /identity/v3/projects => generated 371 bytes in 403 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:10:49.324052 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d47827-58ec-4267-bb79-0eace56e0137 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:49.325364 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dfef76-e0a0-4dc2-85a5-f4dd2d453a74 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:49.326114 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-61d47827-58ec-4267-bb79-0eace56e0137 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:49.340891 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d47827-58ec-4267-bb79-0eace56e0137 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:49.397605 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dfef76-e0a0-4dc2-85a5-f4dd2d453a74 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:49.402520 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0348da-f4a4-4423-b7ce-0d0a45db8bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:49.408003 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0348da-f4a4-4423-b7ce-0d0a45db8bba admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:49.408130 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39dcebe-3971-4b8e-842c-4e5808f9c9db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:49.408234 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0348da-f4a4-4423-b7ce-0d0a45db8bba admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:49.408234 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0348da-f4a4-4423-b7ce-0d0a45db8bba admin admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:49.408368 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39dcebe-3971-4b8e-842c-4e5808f9c9db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:49.408368 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39dcebe-3971-4b8e-842c-4e5808f9c9db admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:49.408570 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0348da-f4a4-4423-b7ce-0d0a45db8bba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:49.408640 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39dcebe-3971-4b8e-842c-4e5808f9c9db admin admin] PATH_INFO: `/v3/projects/96359b60fff44193864fab8bd71cfed8` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:49.408640 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39dcebe-3971-4b8e-842c-4e5808f9c9db admin admin] RBAC: Authorizing `identity:delete_project(project_id=96359b60fff44193864fab8bd71cfed8)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:49.408770 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0348da-f4a4-4423-b7ce-0d0a45db8bba admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:49.408849 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39dcebe-3971-4b8e-842c-4e5808f9c9db admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:49.557639 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 901/1791] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:10:49 2022] POST /identity/v3/users => generated 286 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:10:49.564429 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0348da-f4a4-4423-b7ce-0d0a45db8bba admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:49.570940 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-b39dcebe-3971-4b8e-842c-4e5808f9c9db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96359b60fff44193864fab8bd71cfed8', 'request_id': 'req-b39dcebe-3971-4b8e-842c-4e5808f9c9db'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:49.574615 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0348da-f4a4-4423-b7ce-0d0a45db8bba admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:49.630475 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a09242-888d-460e-98b2-0c6c72255dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:49.631079 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a09242-888d-460e-98b2-0c6c72255dbd admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:49.631362 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a09242-888d-460e-98b2-0c6c72255dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:49.631629 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a09242-888d-460e-98b2-0c6c72255dbd admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:49.632262 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a09242-888d-460e-98b2-0c6c72255dbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:49.633873 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a09242-888d-460e-98b2-0c6c72255dbd admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:49.643712 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 902/1792] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:10:49 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:10:49.647711 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a09242-888d-460e-98b2-0c6c72255dbd admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:49.657546 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a09242-888d-460e-98b2-0c6c72255dbd admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:49.662097 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bff70b-18dd-42b8-9e27-f8581f17bd6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:49.662664 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bff70b-18dd-42b8-9e27-f8581f17bd6f admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:49.662948 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bff70b-18dd-42b8-9e27-f8581f17bd6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:49.663221 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bff70b-18dd-42b8-9e27-f8581f17bd6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:49.663841 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bff70b-18dd-42b8-9e27-f8581f17bd6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:49.665428 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bff70b-18dd-42b8-9e27-f8581f17bd6f admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:49.676196 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 903/1793] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:10:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 22:10:49.681221 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bff70b-18dd-42b8-9e27-f8581f17bd6f admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:49.690143 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bff70b-18dd-42b8-9e27-f8581f17bd6f admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:49.694850 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aed843e-2d2f-44ac-a51d-b297a5baf04f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:49.695453 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aed843e-2d2f-44ac-a51d-b297a5baf04f admin admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:49.695727 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aed843e-2d2f-44ac-a51d-b297a5baf04f admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:49.695995 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aed843e-2d2f-44ac-a51d-b297a5baf04f admin admin] PATH_INFO: `/v3/projects/37de348fc2fa460da46f7160d7f8f85d/users/15cfb59a36f843839b48e200cc3b3efb/roles/56e4a96094e9434cbbafb22579c034f5` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:49.709880 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aed843e-2d2f-44ac-a51d-b297a5baf04f admin admin] RBAC: Authorizing `identity:create_grant(project_id=37de348fc2fa460da46f7160d7f8f85d, user_id=15cfb59a36f843839b48e200cc3b3efb, role_id=56e4a96094e9434cbbafb22579c034f5)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:49.710281 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-b39dcebe-3971-4b8e-842c-4e5808f9c9db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96359b60fff44193864fab8bd71cfed8', 'request_id': 'req-b39dcebe-3971-4b8e-842c-4e5808f9c9db'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:49.712500 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aed843e-2d2f-44ac-a51d-b297a5baf04f admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:49.716390 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-b39dcebe-3971-4b8e-842c-4e5808f9c9db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96359b60fff44193864fab8bd71cfed8', 'request_id': 'req-b39dcebe-3971-4b8e-842c-4e5808f9c9db'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:49.728554 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-b39dcebe-3971-4b8e-842c-4e5808f9c9db admin admin] The token cache is being invalidate because project 96359b60fff44193864fab8bd71cfed8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:10:49.728848 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-b39dcebe-3971-4b8e-842c-4e5808f9c9db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:49.729494 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-b39dcebe-3971-4b8e-842c-4e5808f9c9db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:49.730985 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 891/1794] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:10:49 2022] DELETE /identity/v3/projects/96359b60fff44193864fab8bd71cfed8 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:10:49.735390 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39dcebe-3971-4b8e-842c-4e5808f9c9db admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:49.736793 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-b39dcebe-3971-4b8e-842c-4e5808f9c9db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:49.749780 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39dcebe-3971-4b8e-842c-4e5808f9c9db admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:49.755823 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6cb3ff-bf17-4b6c-820f-437983c9ef59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:49.755823 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6cb3ff-bf17-4b6c-820f-437983c9ef59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:49.755823 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6cb3ff-bf17-4b6c-820f-437983c9ef59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:49.755823 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6cb3ff-bf17-4b6c-820f-437983c9ef59 admin admin] PATH_INFO: `/v3/users/da5ce139eab449eb88d0018feb9a9060` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:49.756838 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6cb3ff-bf17-4b6c-820f-437983c9ef59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da5ce139eab449eb88d0018feb9a9060)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:49.758758 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6cb3ff-bf17-4b6c-820f-437983c9ef59 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:49.835871 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 904/1795] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:10:49 2022] PUT /identity/v3/projects/37de348fc2fa460da46f7160d7f8f85d/users/15cfb59a36f843839b48e200cc3b3efb/roles/56e4a96094e9434cbbafb22579c034f5 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 22:10:49.847871 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276eae44-6104-4cd8-a53c-8a33fcb9cc74 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:49.848158 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276eae44-6104-4cd8-a53c-8a33fcb9cc74 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:49.848460 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276eae44-6104-4cd8-a53c-8a33fcb9cc74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:49.871129 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:10:49.878712 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-276eae44-6104-4cd8-a53c-8a33fcb9cc74 None None] MFA Rules not processed for user `15cfb59a36f843839b48e200cc3b3efb`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:49.880480 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-276eae44-6104-4cd8-a53c-8a33fcb9cc74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:49.943969 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 905/1796] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:10:49 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 22:10:49.948583 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276eae44-6104-4cd8-a53c-8a33fcb9cc74 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:49.949910 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-276eae44-6104-4cd8-a53c-8a33fcb9cc74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:50.015930 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276eae44-6104-4cd8-a53c-8a33fcb9cc74 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:50.020818 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db57f7c-ea44-4803-843d-6ce550c3fc58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:50.021548 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db57f7c-ea44-4803-843d-6ce550c3fc58 admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:50.021835 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db57f7c-ea44-4803-843d-6ce550c3fc58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:50.022108 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db57f7c-ea44-4803-843d-6ce550c3fc58 admin admin] PATH_INFO: `/v3/projects` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:50.022742 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db57f7c-ea44-4803-843d-6ce550c3fc58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:50.024712 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db57f7c-ea44-4803-843d-6ce550c3fc58 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:50.040184 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-0b6cb3ff-bf17-4b6c-820f-437983c9ef59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da5ce139eab449eb88d0018feb9a9060'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:50.054633 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-0b6cb3ff-bf17-4b6c-820f-437983c9ef59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da5ce139eab449eb88d0018feb9a9060'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:50.061630 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-0b6cb3ff-bf17-4b6c-820f-437983c9ef59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da5ce139eab449eb88d0018feb9a9060'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:50.062359 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-0b6cb3ff-bf17-4b6c-820f-437983c9ef59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da5ce139eab449eb88d0018feb9a9060'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:50.063019 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-0b6cb3ff-bf17-4b6c-820f-437983c9ef59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da5ce139eab449eb88d0018feb9a9060'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:50.216533 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 892/1797] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:10:49 2022] DELETE /identity/v3/users/da5ce139eab449eb88d0018feb9a9060 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 22:10:50.216870 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 906/1798] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:10:49 2022] POST /identity/v3/projects => generated 373 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 22:10:50.219880 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6cb3ff-bf17-4b6c-820f-437983c9ef59 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:50.221398 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-0b6cb3ff-bf17-4b6c-820f-437983c9ef59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:50.320442 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6cb3ff-bf17-4b6c-820f-437983c9ef59 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:50.393457 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85089722-d649-466f-9824-2606c21b1612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:50.394226 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85089722-d649-466f-9824-2606c21b1612 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:50.394600 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85089722-d649-466f-9824-2606c21b1612 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:50.394935 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85089722-d649-466f-9824-2606c21b1612 admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:50.395632 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85089722-d649-466f-9824-2606c21b1612 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:50.397778 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85089722-d649-466f-9824-2606c21b1612 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:50.533791 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 893/1799] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:10:50 2022] POST /identity/v3/users => generated 293 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:10:50.537783 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db57f7c-ea44-4803-843d-6ce550c3fc58 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:50.549806 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db57f7c-ea44-4803-843d-6ce550c3fc58 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:50.554463 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0214c1-00e3-4c82-b960-5f1609e6614b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:50.555077 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0214c1-00e3-4c82-b960-5f1609e6614b admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:50.555521 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0214c1-00e3-4c82-b960-5f1609e6614b admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:50.555844 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0214c1-00e3-4c82-b960-5f1609e6614b admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:50.556532 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0214c1-00e3-4c82-b960-5f1609e6614b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:50.558202 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0214c1-00e3-4c82-b960-5f1609e6614b admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:50.569660 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 907/1800] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:10:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:10:50.573760 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85089722-d649-466f-9824-2606c21b1612 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:50.584308 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85089722-d649-466f-9824-2606c21b1612 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:50.591157 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2364e73b-e6ee-4a39-aabd-dc06dc8eab53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:50.592324 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2364e73b-e6ee-4a39-aabd-dc06dc8eab53 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:50.592641 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2364e73b-e6ee-4a39-aabd-dc06dc8eab53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:50.592915 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2364e73b-e6ee-4a39-aabd-dc06dc8eab53 admin admin] PATH_INFO: `/v3/projects/1e947edcc38b45ffa9816a7b3fae80d9/users/c90963e2c575477592f86b5dc06d931d/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:50.614489 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2364e73b-e6ee-4a39-aabd-dc06dc8eab53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e947edcc38b45ffa9816a7b3fae80d9, user_id=c90963e2c575477592f86b5dc06d931d, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:50.617428 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2364e73b-e6ee-4a39-aabd-dc06dc8eab53 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:50.712819 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0214c1-00e3-4c82-b960-5f1609e6614b admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:50.717580 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-5d0214c1-00e3-4c82-b960-5f1609e6614b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:50.747669 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 894/1801] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:10:50 2022] PUT /identity/v3/projects/1e947edcc38b45ffa9816a7b3fae80d9/users/c90963e2c575477592f86b5dc06d931d/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:10:50.749599 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2364e73b-e6ee-4a39-aabd-dc06dc8eab53 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:50.751223 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0214c1-00e3-4c82-b960-5f1609e6614b admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:50.830195 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d363e1-7216-4eb9-bb7c-863bc927d93f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:50.831227 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d363e1-7216-4eb9-bb7c-863bc927d93f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:50.831635 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d363e1-7216-4eb9-bb7c-863bc927d93f admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:50.832068 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d363e1-7216-4eb9-bb7c-863bc927d93f admin admin] PATH_INFO: `/v3/projects/50b87f329654425597e3b41eea4e3dda` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:50.833464 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d363e1-7216-4eb9-bb7c-863bc927d93f admin admin] RBAC: Authorizing `identity:delete_project(project_id=50b87f329654425597e3b41eea4e3dda)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:50.835782 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d363e1-7216-4eb9-bb7c-863bc927d93f admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:50.842414 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2364e73b-e6ee-4a39-aabd-dc06dc8eab53 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:50.847454 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6913883c-c535-4bed-9a58-86a81a1feea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:50.849707 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6913883c-c535-4bed-9a58-86a81a1feea4 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:50.849707 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6913883c-c535-4bed-9a58-86a81a1feea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:50.849707 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6913883c-c535-4bed-9a58-86a81a1feea4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:50.849707 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6913883c-c535-4bed-9a58-86a81a1feea4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:50.851316 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6913883c-c535-4bed-9a58-86a81a1feea4 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:50.871744 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 895/1802] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:10:50 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 22:10:50.874010 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6913883c-c535-4bed-9a58-86a81a1feea4 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:50.885566 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6913883c-c535-4bed-9a58-86a81a1feea4 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:50.890854 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e45cc43-192f-40f3-9bcf-c72fc0cedab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:50.891708 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e45cc43-192f-40f3-9bcf-c72fc0cedab9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:50.892367 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e45cc43-192f-40f3-9bcf-c72fc0cedab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:50.892675 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e45cc43-192f-40f3-9bcf-c72fc0cedab9 admin admin] PATH_INFO: `/v3/projects/1e947edcc38b45ffa9816a7b3fae80d9/users/c90963e2c575477592f86b5dc06d931d/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:50.894624 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e45cc43-192f-40f3-9bcf-c72fc0cedab9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e947edcc38b45ffa9816a7b3fae80d9, user_id=c90963e2c575477592f86b5dc06d931d, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:50.897525 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e45cc43-192f-40f3-9bcf-c72fc0cedab9 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:50.909496 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 896/1803] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:10:50 2022] PUT /identity/v3/projects/1e947edcc38b45ffa9816a7b3fae80d9/users/c90963e2c575477592f86b5dc06d931d/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:10:50.920916 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069d5d60-2358-487c-9b66-c8624d3f5a1d None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:50.921204 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069d5d60-2358-487c-9b66-c8624d3f5a1d None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:50.921477 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069d5d60-2358-487c-9b66-c8624d3f5a1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:50.955744 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-069d5d60-2358-487c-9b66-c8624d3f5a1d None None] MFA Rules not processed for user `c90963e2c575477592f86b5dc06d931d`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:50.957089 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-069d5d60-2358-487c-9b66-c8624d3f5a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:50.962098 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-75d363e1-7216-4eb9-bb7c-863bc927d93f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50b87f329654425597e3b41eea4e3dda', 'request_id': 'req-75d363e1-7216-4eb9-bb7c-863bc927d93f'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:51.039172 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 897/1804] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:10:50 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:10:51.042003 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-75d363e1-7216-4eb9-bb7c-863bc927d93f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50b87f329654425597e3b41eea4e3dda', 'request_id': 'req-75d363e1-7216-4eb9-bb7c-863bc927d93f'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:51.049621 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-75d363e1-7216-4eb9-bb7c-863bc927d93f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50b87f329654425597e3b41eea4e3dda', 'request_id': 'req-75d363e1-7216-4eb9-bb7c-863bc927d93f'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:51.057239 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069d5d60-2358-487c-9b66-c8624d3f5a1d None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:51.058911 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-069d5d60-2358-487c-9b66-c8624d3f5a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:51.074054 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-75d363e1-7216-4eb9-bb7c-863bc927d93f admin admin] The token cache is being invalidate because project 50b87f329654425597e3b41eea4e3dda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:10:51.074331 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-75d363e1-7216-4eb9-bb7c-863bc927d93f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:51.075006 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-75d363e1-7216-4eb9-bb7c-863bc927d93f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:51.087562 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 908/1805] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:10:50 2022] DELETE /identity/v3/projects/50b87f329654425597e3b41eea4e3dda => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 22:10:51.093937 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e863c22b-c9bd-414b-9f47-dc92b2714608 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:51.094232 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e863c22b-c9bd-414b-9f47-dc92b2714608 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:51.094569 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e863c22b-c9bd-414b-9f47-dc92b2714608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:51.124108 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-e863c22b-c9bd-414b-9f47-dc92b2714608 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:51.127786 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-e863c22b-c9bd-414b-9f47-dc92b2714608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:51.132100 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069d5d60-2358-487c-9b66-c8624d3f5a1d None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:51.136915 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d94c1b-172b-47d6-b72c-ed16b692fd0d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa71c71de263478da234f3c5716a6e5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:51.137651 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d94c1b-172b-47d6-b72c-ed16b692fd0d service cinder] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:51.138148 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d94c1b-172b-47d6-b72c-ed16b692fd0d service cinder] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:51.138593 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d94c1b-172b-47d6-b72c-ed16b692fd0d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:51.140725 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-29d94c1b-172b-47d6-b72c-ed16b692fd0d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:51.180090 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d94c1b-172b-47d6-b72c-ed16b692fd0d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:51.184262 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d94c1b-172b-47d6-b72c-ed16b692fd0d service cinder] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:51.200856 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 909/1806] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:10:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:10:51.205909 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 898/1807] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:10:51 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:10:51.213173 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a063da-4d4d-4206-aeb2-6dce7daff76b None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:51.215601 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a063da-4d4d-4206-aeb2-6dce7daff76b None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:51.215884 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a063da-4d4d-4206-aeb2-6dce7daff76b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:51.256516 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-a4a063da-4d4d-4206-aeb2-6dce7daff76b None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:51.258565 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-a4a063da-4d4d-4206-aeb2-6dce7daff76b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:51.276880 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 910/1808] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:10:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:10:51.282866 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d94c1b-172b-47d6-b72c-ed16b692fd0d service cinder] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:51.292590 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d94c1b-172b-47d6-b72c-ed16b692fd0d service cinder] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:51.298943 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0ab4a8-05fb-4ae6-a286-649c379fded1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:51.299623 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0ab4a8-05fb-4ae6-a286-649c379fded1 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:51.299973 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0ab4a8-05fb-4ae6-a286-649c379fded1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:51.300308 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0ab4a8-05fb-4ae6-a286-649c379fded1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:51.301045 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0ab4a8-05fb-4ae6-a286-649c379fded1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 22:10:51.301417 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0ab4a8-05fb-4ae6-a286-649c379fded1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:51.304457 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0ab4a8-05fb-4ae6-a286-649c379fded1 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:51.322240 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 899/1809] 10.222.0.18 () {58 vars in 1153 bytes} [Sat Aug 6 22:10:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 22:10:51.326415 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a063da-4d4d-4206-aeb2-6dce7daff76b None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:51.343544 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a063da-4d4d-4206-aeb2-6dce7daff76b None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:51.366603 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:10:51.370700 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93486aae-26bc-4aad-bb1b-7afe1c92b751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:51.372750 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93486aae-26bc-4aad-bb1b-7afe1c92b751 admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:51.373660 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93486aae-26bc-4aad-bb1b-7afe1c92b751 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:51.374600 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93486aae-26bc-4aad-bb1b-7afe1c92b751 admin admin] PATH_INFO: `/v3/projects` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:51.380605 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93486aae-26bc-4aad-bb1b-7afe1c92b751 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:51.386232 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93486aae-26bc-4aad-bb1b-7afe1c92b751 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:51.540674 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 911/1810] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:10:51 2022] POST /identity/v3/projects => generated 383 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:10:51.541920 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0ab4a8-05fb-4ae6-a286-649c379fded1 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:51.558243 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0ab4a8-05fb-4ae6-a286-649c379fded1 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:51.642967 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a532a3-1fff-4230-9814-3a089a27153b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:51.643499 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a532a3-1fff-4230-9814-3a089a27153b admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:51.643780 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a532a3-1fff-4230-9814-3a089a27153b admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:51.644054 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a532a3-1fff-4230-9814-3a089a27153b admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:51.644716 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a532a3-1fff-4230-9814-3a089a27153b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:51.646885 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a532a3-1fff-4230-9814-3a089a27153b admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:51.742447 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 900/1811] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:10:51 2022] POST /identity/v3/users => generated 292 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:10:51.745460 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93486aae-26bc-4aad-bb1b-7afe1c92b751 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:51.755736 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93486aae-26bc-4aad-bb1b-7afe1c92b751 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:51.763347 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd69cafc-e4c9-42b1-9b5e-3b734a4ebc59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:51.764715 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd69cafc-e4c9-42b1-9b5e-3b734a4ebc59 admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:51.765157 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd69cafc-e4c9-42b1-9b5e-3b734a4ebc59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:51.765426 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd69cafc-e4c9-42b1-9b5e-3b734a4ebc59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:51.766067 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd69cafc-e4c9-42b1-9b5e-3b734a4ebc59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:51.767646 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd69cafc-e4c9-42b1-9b5e-3b734a4ebc59 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:51.777952 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 912/1812] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:10:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:10:51.782417 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a532a3-1fff-4230-9814-3a089a27153b admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:51.793013 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a532a3-1fff-4230-9814-3a089a27153b admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:51.798461 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08685f0-0261-46bc-9ccd-746e5062c3f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:51.799084 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08685f0-0261-46bc-9ccd-746e5062c3f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:51.799523 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08685f0-0261-46bc-9ccd-746e5062c3f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:51.799800 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08685f0-0261-46bc-9ccd-746e5062c3f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:51.800472 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08685f0-0261-46bc-9ccd-746e5062c3f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:51.802107 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08685f0-0261-46bc-9ccd-746e5062c3f9 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:51.813407 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 901/1813] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:10:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:10:51.818006 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd69cafc-e4c9-42b1-9b5e-3b734a4ebc59 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:51.828651 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd69cafc-e4c9-42b1-9b5e-3b734a4ebc59 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:51.833093 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ec96a5-1262-4bca-9b83-a1f290e95f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:51.833730 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ec96a5-1262-4bca-9b83-a1f290e95f73 admin admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:51.834007 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ec96a5-1262-4bca-9b83-a1f290e95f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:51.834276 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ec96a5-1262-4bca-9b83-a1f290e95f73 admin admin] PATH_INFO: `/v3/projects/540fabe5da6d4928a4411e4c96cb4256/users/88b5e81665d84915ab400ebe8f9f1c61/roles/56e4a96094e9434cbbafb22579c034f5` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:51.849488 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ec96a5-1262-4bca-9b83-a1f290e95f73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=540fabe5da6d4928a4411e4c96cb4256, user_id=88b5e81665d84915ab400ebe8f9f1c61, role_id=56e4a96094e9434cbbafb22579c034f5)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:51.852416 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ec96a5-1262-4bca-9b83-a1f290e95f73 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:51.985444 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 913/1814] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:10:51 2022] PUT /identity/v3/projects/540fabe5da6d4928a4411e4c96cb4256/users/88b5e81665d84915ab400ebe8f9f1c61/roles/56e4a96094e9434cbbafb22579c034f5 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:10:51.995817 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee45942-ed62-44f1-bb19-703920572a7f None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:51.996094 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee45942-ed62-44f1-bb19-703920572a7f None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:51.996364 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee45942-ed62-44f1-bb19-703920572a7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:52.026719 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-8ee45942-ed62-44f1-bb19-703920572a7f None None] MFA Rules not processed for user `88b5e81665d84915ab400ebe8f9f1c61`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:52.027958 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-8ee45942-ed62-44f1-bb19-703920572a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:52.107137 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 902/1815] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:10:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 22:10:52.112530 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ec96a5-1262-4bca-9b83-a1f290e95f73 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:52.186839 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ec96a5-1262-4bca-9b83-a1f290e95f73 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:52.191434 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b2ee94-41a9-49e0-b422-d9ee33d519d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:52.192034 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b2ee94-41a9-49e0-b422-d9ee33d519d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:52.192310 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b2ee94-41a9-49e0-b422-d9ee33d519d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:52.192637 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b2ee94-41a9-49e0-b422-d9ee33d519d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:52.193293 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b2ee94-41a9-49e0-b422-d9ee33d519d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:52.195232 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b2ee94-41a9-49e0-b422-d9ee33d519d5 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:52.289832 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 914/1816] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:10:52 2022] POST /identity/v3/projects => generated 381 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:10:52.293937 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee45942-ed62-44f1-bb19-703920572a7f None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:52.311759 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee45942-ed62-44f1-bb19-703920572a7f None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:52.391041 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f488db01-8e13-42e2-b07f-eb98c15a3dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:52.392501 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f488db01-8e13-42e2-b07f-eb98c15a3dc1 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:52.392501 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f488db01-8e13-42e2-b07f-eb98c15a3dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:52.392501 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f488db01-8e13-42e2-b07f-eb98c15a3dc1 admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:52.392796 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f488db01-8e13-42e2-b07f-eb98c15a3dc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:52.394711 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f488db01-8e13-42e2-b07f-eb98c15a3dc1 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:52.447303 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b2ee94-41a9-49e0-b422-d9ee33d519d5 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:52.449968 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-b4b2ee94-41a9-49e0-b422-d9ee33d519d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:52.462590 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b2ee94-41a9-49e0-b422-d9ee33d519d5 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:52.467880 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea206911-637c-4cc7-b6ab-de0bcef38b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:52.468553 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea206911-637c-4cc7-b6ab-de0bcef38b90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:52.468839 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea206911-637c-4cc7-b6ab-de0bcef38b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:52.469174 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea206911-637c-4cc7-b6ab-de0bcef38b90 admin admin] PATH_INFO: `/v3/users/15cfb59a36f843839b48e200cc3b3efb` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:52.470363 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea206911-637c-4cc7-b6ab-de0bcef38b90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15cfb59a36f843839b48e200cc3b3efb)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:52.472354 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea206911-637c-4cc7-b6ab-de0bcef38b90 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:52.493021 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 903/1817] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:10:52 2022] POST /identity/v3/users => generated 297 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:10:52.494471 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f488db01-8e13-42e2-b07f-eb98c15a3dc1 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:52.510887 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f488db01-8e13-42e2-b07f-eb98c15a3dc1 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:52.519205 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4950b183-df2c-4d7b-adaa-866d29bbfa0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:52.520138 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4950b183-df2c-4d7b-adaa-866d29bbfa0d admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:52.520628 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4950b183-df2c-4d7b-adaa-866d29bbfa0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:52.521098 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4950b183-df2c-4d7b-adaa-866d29bbfa0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:52.523099 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4950b183-df2c-4d7b-adaa-866d29bbfa0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:52.525963 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4950b183-df2c-4d7b-adaa-866d29bbfa0d admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:52.545615 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 904/1818] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:10:52 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:10:52.550464 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4950b183-df2c-4d7b-adaa-866d29bbfa0d admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:52.566455 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4950b183-df2c-4d7b-adaa-866d29bbfa0d admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:52.581838 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1444c478-d413-4b0f-bc85-e5306291b3f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:52.583002 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1444c478-d413-4b0f-bc85-e5306291b3f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:52.583498 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1444c478-d413-4b0f-bc85-e5306291b3f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:52.583998 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1444c478-d413-4b0f-bc85-e5306291b3f8 admin admin] PATH_INFO: `/v3/projects/c5db62acdba04a64b5dc8d09add306bc/users/94de72fe0e5047cb95ce0ac27d86b23e/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:52.601744 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1444c478-d413-4b0f-bc85-e5306291b3f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5db62acdba04a64b5dc8d09add306bc, user_id=94de72fe0e5047cb95ce0ac27d86b23e, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:52.604607 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1444c478-d413-4b0f-bc85-e5306291b3f8 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:52.680263 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-ea206911-637c-4cc7-b6ab-de0bcef38b90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15cfb59a36f843839b48e200cc3b3efb'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:52.694575 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 905/1819] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:10:52 2022] PUT /identity/v3/projects/c5db62acdba04a64b5dc8d09add306bc/users/94de72fe0e5047cb95ce0ac27d86b23e/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:10:52.698576 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1444c478-d413-4b0f-bc85-e5306291b3f8 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:52.704576 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-ea206911-637c-4cc7-b6ab-de0bcef38b90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15cfb59a36f843839b48e200cc3b3efb'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:52.808138 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:10:52.828290 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1444c478-d413-4b0f-bc85-e5306291b3f8 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:52.833379 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd348c9-00ce-4b25-927f-4c8da657461f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:52.834010 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd348c9-00ce-4b25-927f-4c8da657461f admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:52.834294 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd348c9-00ce-4b25-927f-4c8da657461f admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:52.834750 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd348c9-00ce-4b25-927f-4c8da657461f admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:52.835737 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd348c9-00ce-4b25-927f-4c8da657461f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:52.837549 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd348c9-00ce-4b25-927f-4c8da657461f admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:52.844157 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-ea206911-637c-4cc7-b6ab-de0bcef38b90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15cfb59a36f843839b48e200cc3b3efb'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:52.854884 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-ea206911-637c-4cc7-b6ab-de0bcef38b90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15cfb59a36f843839b48e200cc3b3efb'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:52.859287 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-ea206911-637c-4cc7-b6ab-de0bcef38b90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15cfb59a36f843839b48e200cc3b3efb'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:52.863496 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 906/1820] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:10:52 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 22:10:52.864828 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd348c9-00ce-4b25-927f-4c8da657461f admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:52.867966 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-ecd348c9-00ce-4b25-927f-4c8da657461f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:52.872819 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 915/1821] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:10:52 2022] DELETE /identity/v3/users/15cfb59a36f843839b48e200cc3b3efb => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 22:10:52.877999 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6620ba9b-4ed1-4637-a81e-ab760ed37f16 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:52.878607 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6620ba9b-4ed1-4637-a81e-ab760ed37f16 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:52.879191 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6620ba9b-4ed1-4637-a81e-ab760ed37f16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:52.893813 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd348c9-00ce-4b25-927f-4c8da657461f admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:52.913599 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-6620ba9b-4ed1-4637-a81e-ab760ed37f16 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:52.915560 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-6620ba9b-4ed1-4637-a81e-ab760ed37f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:52.961114 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7724332e-ff62-439a-a543-dd2fc7b18393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:52.962828 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7724332e-ff62-439a-a543-dd2fc7b18393 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:52.963334 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7724332e-ff62-439a-a543-dd2fc7b18393 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:52.963794 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7724332e-ff62-439a-a543-dd2fc7b18393 admin admin] PATH_INFO: `/v3/projects/c5db62acdba04a64b5dc8d09add306bc/users/94de72fe0e5047cb95ce0ac27d86b23e/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:52.966139 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7724332e-ff62-439a-a543-dd2fc7b18393 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5db62acdba04a64b5dc8d09add306bc, user_id=94de72fe0e5047cb95ce0ac27d86b23e, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:52.970143 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7724332e-ff62-439a-a543-dd2fc7b18393 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:52.994814 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 907/1822] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:10:52 2022] PUT /identity/v3/projects/c5db62acdba04a64b5dc8d09add306bc/users/94de72fe0e5047cb95ce0ac27d86b23e/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:10:53.000884 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef41759f-339e-4d11-8002-85cded766e43 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:53.001169 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef41759f-339e-4d11-8002-85cded766e43 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:53.001455 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef41759f-339e-4d11-8002-85cded766e43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:53.049878 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-ef41759f-339e-4d11-8002-85cded766e43 None None] MFA Rules not processed for user `94de72fe0e5047cb95ce0ac27d86b23e`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:53.050015 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-ef41759f-339e-4d11-8002-85cded766e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:53.078603 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 916/1823] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:10:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:10:53.092030 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6620ba9b-4ed1-4637-a81e-ab760ed37f16 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:53.093973 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-6620ba9b-4ed1-4637-a81e-ab760ed37f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:53.180940 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6620ba9b-4ed1-4637-a81e-ab760ed37f16 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:53.187512 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 908/1824] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:10:52 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:10:53.208438 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef41759f-339e-4d11-8002-85cded766e43 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:53.208438 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-ef41759f-339e-4d11-8002-85cded766e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:53.279559 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a89e41e-f538-4b1d-82ca-70fe4fab70d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:53.279559 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a89e41e-f538-4b1d-82ca-70fe4fab70d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:53.279559 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a89e41e-f538-4b1d-82ca-70fe4fab70d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:53.279559 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a89e41e-f538-4b1d-82ca-70fe4fab70d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:53.288444 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef41759f-339e-4d11-8002-85cded766e43 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:53.292047 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57079da-97dc-4efd-a623-69375381c093 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa71c71de263478da234f3c5716a6e5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:53.292708 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57079da-97dc-4efd-a623-69375381c093 service cinder] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:53.292970 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57079da-97dc-4efd-a623-69375381c093 service cinder] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:53.293238 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57079da-97dc-4efd-a623-69375381c093 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:53.303256 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57079da-97dc-4efd-a623-69375381c093 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:53.304781 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57079da-97dc-4efd-a623-69375381c093 service cinder] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:53.316460 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 909/1825] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:10:53 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 22:10:53.369087 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a89e41e-f538-4b1d-82ca-70fe4fab70d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:53.374505 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a89e41e-f538-4b1d-82ca-70fe4fab70d2 service neutron] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:53.397976 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 917/1826] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:10:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 22:10:53.460862 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57079da-97dc-4efd-a623-69375381c093 service cinder] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:53.466260 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57079da-97dc-4efd-a623-69375381c093 service cinder] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:53.474208 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a89e41e-f538-4b1d-82ca-70fe4fab70d2 service neutron] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:53.479581 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-1a89e41e-f538-4b1d-82ca-70fe4fab70d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:53.484525 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa8a551-c18a-435c-a89f-c019216a4dc5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa71c71de263478da234f3c5716a6e5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:53.486692 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa8a551-c18a-435c-a89f-c019216a4dc5 service cinder] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:53.487897 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa8a551-c18a-435c-a89f-c019216a4dc5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:53.488506 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa8a551-c18a-435c-a89f-c019216a4dc5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:53.494036 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-9fa8a551-c18a-435c-a89f-c019216a4dc5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:53.502727 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a89e41e-f538-4b1d-82ca-70fe4fab70d2 service neutron] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:53.511450 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e3e522-41bc-4c0c-a095-70d8833fc306 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa71c71de263478da234f3c5716a6e5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:53.513087 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e3e522-41bc-4c0c-a095-70d8833fc306 service cinder] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:53.515242 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e3e522-41bc-4c0c-a095-70d8833fc306 service cinder] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:53.515748 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e3e522-41bc-4c0c-a095-70d8833fc306 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:53.521529 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-c4e3e522-41bc-4c0c-a095-70d8833fc306 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:53.630889 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa8a551-c18a-435c-a89f-c019216a4dc5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:53.632390 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa8a551-c18a-435c-a89f-c019216a4dc5 service cinder] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:53.647892 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 910/1827] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:10:53 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:10:53.650893 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e3e522-41bc-4c0c-a095-70d8833fc306 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:53.652588 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e3e522-41bc-4c0c-a095-70d8833fc306 service cinder] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:53.672185 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 918/1828] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:10:53 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:10:54.256481 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa8a551-c18a-435c-a89f-c019216a4dc5 service cinder] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:54.258107 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-9fa8a551-c18a-435c-a89f-c019216a4dc5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:54.276880 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e3e522-41bc-4c0c-a095-70d8833fc306 service cinder] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:54.284778 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa8a551-c18a-435c-a89f-c019216a4dc5 service cinder] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:54.292743 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:10:54.302702 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4bacdf-a8d4-494c-84f2-ded7b9ef3252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:54.303415 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4bacdf-a8d4-494c-84f2-ded7b9ef3252 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:54.303716 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4bacdf-a8d4-494c-84f2-ded7b9ef3252 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:54.304000 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4bacdf-a8d4-494c-84f2-ded7b9ef3252 admin admin] PATH_INFO: `/v3/projects/37de348fc2fa460da46f7160d7f8f85d` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:54.310516 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4bacdf-a8d4-494c-84f2-ded7b9ef3252 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37de348fc2fa460da46f7160d7f8f85d)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:54.315997 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4bacdf-a8d4-494c-84f2-ded7b9ef3252 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:54.325510 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e3e522-41bc-4c0c-a095-70d8833fc306 service cinder] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:54.331287 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7ed9a0-826a-40da-b596-2c96a296c2c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:54.332673 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ed9a0-826a-40da-b596-2c96a296c2c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:54.333158 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ed9a0-826a-40da-b596-2c96a296c2c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:54.333881 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ed9a0-826a-40da-b596-2c96a296c2c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:54.336121 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-cb7ed9a0-826a-40da-b596-2c96a296c2c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:54.357062 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7ed9a0-826a-40da-b596-2c96a296c2c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:54.360293 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7ed9a0-826a-40da-b596-2c96a296c2c2 service neutron] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:54.402197 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 919/1829] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:10:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:10:54.403053 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7ed9a0-826a-40da-b596-2c96a296c2c2 service neutron] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:54.430040 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7ed9a0-826a-40da-b596-2c96a296c2c2 service neutron] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:54.443204 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbfc042-bb65-4144-90e7-dd75af0687dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:54.444118 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbfc042-bb65-4144-90e7-dd75af0687dd service neutron] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:54.444982 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbfc042-bb65-4144-90e7-dd75af0687dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:54.445271 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbfc042-bb65-4144-90e7-dd75af0687dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:54.470055 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbfc042-bb65-4144-90e7-dd75af0687dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:54.471515 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbfc042-bb65-4144-90e7-dd75af0687dd service neutron] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:54.495701 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 920/1830] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:10:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:10:54.500450 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbfc042-bb65-4144-90e7-dd75af0687dd service neutron] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:54.504348 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-5c4bacdf-a8d4-494c-84f2-ded7b9ef3252 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37de348fc2fa460da46f7160d7f8f85d', 'request_id': 'req-5c4bacdf-a8d4-494c-84f2-ded7b9ef3252'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:54.504621 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-dbbfc042-bb65-4144-90e7-dd75af0687dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:54.521403 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbfc042-bb65-4144-90e7-dd75af0687dd service neutron] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:54.590631 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd2017e-6c65-44d1-9453-4df8818b15a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:54.591726 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd2017e-6c65-44d1-9453-4df8818b15a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:54.592218 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd2017e-6c65-44d1-9453-4df8818b15a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:54.593727 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd2017e-6c65-44d1-9453-4df8818b15a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:54.604665 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd2017e-6c65-44d1-9453-4df8818b15a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:54.606930 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd2017e-6c65-44d1-9453-4df8818b15a7 service neutron] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:54.608342 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-5c4bacdf-a8d4-494c-84f2-ded7b9ef3252 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37de348fc2fa460da46f7160d7f8f85d', 'request_id': 'req-5c4bacdf-a8d4-494c-84f2-ded7b9ef3252'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:54.617536 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 921/1831] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:10:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:10:54.618298 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-5c4bacdf-a8d4-494c-84f2-ded7b9ef3252 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37de348fc2fa460da46f7160d7f8f85d', 'request_id': 'req-5c4bacdf-a8d4-494c-84f2-ded7b9ef3252'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:54.647500 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-5c4bacdf-a8d4-494c-84f2-ded7b9ef3252 admin admin] The token cache is being invalidate because project 37de348fc2fa460da46f7160d7f8f85d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:10:54.648390 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-5c4bacdf-a8d4-494c-84f2-ded7b9ef3252 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:54.649587 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-5c4bacdf-a8d4-494c-84f2-ded7b9ef3252 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:54.652061 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 911/1832] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:10:54 2022] DELETE /identity/v3/projects/37de348fc2fa460da46f7160d7f8f85d => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:10:54.658916 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd2017e-6c65-44d1-9453-4df8818b15a7 service neutron] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:54.823393 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd2017e-6c65-44d1-9453-4df8818b15a7 service neutron] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:54.829603 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5a53e1-cd1c-412e-9a1f-f165e668e087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:54.830339 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a53e1-cd1c-412e-9a1f-f165e668e087 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:54.830686 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a53e1-cd1c-412e-9a1f-f165e668e087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:54.831027 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a53e1-cd1c-412e-9a1f-f165e668e087 admin admin] PATH_INFO: `/v3/users/c90963e2c575477592f86b5dc06d931d` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:54.832267 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5a53e1-cd1c-412e-9a1f-f165e668e087 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c90963e2c575477592f86b5dc06d931d)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:54.835280 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5a53e1-cd1c-412e-9a1f-f165e668e087 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:55.105495 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:10:55.186415 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-ba5a53e1-cd1c-412e-9a1f-f165e668e087 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c90963e2c575477592f86b5dc06d931d'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:55.243489 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-ba5a53e1-cd1c-412e-9a1f-f165e668e087 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c90963e2c575477592f86b5dc06d931d'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:55.386505 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-ba5a53e1-cd1c-412e-9a1f-f165e668e087 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c90963e2c575477592f86b5dc06d931d'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:55.400477 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-ba5a53e1-cd1c-412e-9a1f-f165e668e087 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c90963e2c575477592f86b5dc06d931d'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:55.402994 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-ba5a53e1-cd1c-412e-9a1f-f165e668e087 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c90963e2c575477592f86b5dc06d931d'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:55.439049 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 922/1833] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:10:54 2022] DELETE /identity/v3/users/c90963e2c575477592f86b5dc06d931d => generated 0 bytes in 776 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:10:55.450678 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4bacdf-a8d4-494c-84f2-ded7b9ef3252 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:55.452950 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-5c4bacdf-a8d4-494c-84f2-ded7b9ef3252 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:55.473221 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4bacdf-a8d4-494c-84f2-ded7b9ef3252 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:55.604139 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a53e953-8b84-476a-8c2c-717e1946f138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:55.604852 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a53e953-8b84-476a-8c2c-717e1946f138 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:55.605194 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a53e953-8b84-476a-8c2c-717e1946f138 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:55.605478 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a53e953-8b84-476a-8c2c-717e1946f138 admin admin] PATH_INFO: `/v3/users/8750d883fc8747a29dc1051cfaa0c5f6` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:55.610902 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a53e953-8b84-476a-8c2c-717e1946f138 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8750d883fc8747a29dc1051cfaa0c5f6)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:55.613045 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a53e953-8b84-476a-8c2c-717e1946f138 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:55.814916 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-2a53e953-8b84-476a-8c2c-717e1946f138 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8750d883fc8747a29dc1051cfaa0c5f6'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:55.833332 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-2a53e953-8b84-476a-8c2c-717e1946f138 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8750d883fc8747a29dc1051cfaa0c5f6'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:55.843895 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-2a53e953-8b84-476a-8c2c-717e1946f138 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8750d883fc8747a29dc1051cfaa0c5f6'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:55.846649 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-2a53e953-8b84-476a-8c2c-717e1946f138 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8750d883fc8747a29dc1051cfaa0c5f6'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:55.846649 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-2a53e953-8b84-476a-8c2c-717e1946f138 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8750d883fc8747a29dc1051cfaa0c5f6'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:55.927897 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 912/1834] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:10:55 2022] DELETE /identity/v3/users/8750d883fc8747a29dc1051cfaa0c5f6 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 22:10:55.929714 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976d2a8d-4679-42b9-9cf3-cb4977fdbc81 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:55.930055 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976d2a8d-4679-42b9-9cf3-cb4977fdbc81 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:55.930246 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976d2a8d-4679-42b9-9cf3-cb4977fdbc81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:55.962942 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-976d2a8d-4679-42b9-9cf3-cb4977fdbc81 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:55.971527 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-976d2a8d-4679-42b9-9cf3-cb4977fdbc81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:56.094602 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 923/1835] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:10:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:10:56.111517 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a53e953-8b84-476a-8c2c-717e1946f138 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:56.113031 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-2a53e953-8b84-476a-8c2c-717e1946f138 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:56.237185 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a53e953-8b84-476a-8c2c-717e1946f138 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:56.241758 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e1038f-96b6-417f-a7c6-ca0392c4d4a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:56.242671 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e1038f-96b6-417f-a7c6-ca0392c4d4a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:56.253943 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e1038f-96b6-417f-a7c6-ca0392c4d4a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:56.254360 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e1038f-96b6-417f-a7c6-ca0392c4d4a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:56.266874 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e1038f-96b6-417f-a7c6-ca0392c4d4a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:56.268453 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e1038f-96b6-417f-a7c6-ca0392c4d4a1 service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:56.297500 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 913/1836] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:10:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:10:56.297768 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f2307b-5fbf-4d8f-b1cc-c427b7bdeb5b None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:56.297768 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f2307b-5fbf-4d8f-b1cc-c427b7bdeb5b None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:56.297768 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f2307b-5fbf-4d8f-b1cc-c427b7bdeb5b None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:56.298400 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 924/1837] 10.222.0.18 () {54 vars in 921 bytes} [Sat Aug 6 22:10:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:10:56.363958 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56321be0-5a81-4ad4-9d0b-2b07ec092d1a None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:56.364601 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56321be0-5a81-4ad4-9d0b-2b07ec092d1a None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:56.365029 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56321be0-5a81-4ad4-9d0b-2b07ec092d1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:56.431839 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-56321be0-5a81-4ad4-9d0b-2b07ec092d1a None None] MFA Rules not processed for user `2a1319d33e9144859c6010843be312a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:56.431839 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-56321be0-5a81-4ad4-9d0b-2b07ec092d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:56.659004 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 914/1838] 10.222.0.18 () {58 vars in 1018 bytes} [Sat Aug 6 22:10:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:10:56.679419 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f2307b-5fbf-4d8f-b1cc-c427b7bdeb5b None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:56.683139 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-83f2307b-5fbf-4d8f-b1cc-c427b7bdeb5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:56.706672 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88723652-6039-4147-8fc2-3f75a9d672b3 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:56.707163 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88723652-6039-4147-8fc2-3f75a9d672b3 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:56.707440 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88723652-6039-4147-8fc2-3f75a9d672b3 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:56.709814 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 915/1839] 10.222.0.18 () {54 vars in 921 bytes} [Sat Aug 6 22:10:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:10:56.715569 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f2307b-5fbf-4d8f-b1cc-c427b7bdeb5b None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:56.722085 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580edf32-733a-4c59-939c-2e7c202b3639 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a1319d33e9144859c6010843be312a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:56.722969 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580edf32-733a-4c59-939c-2e7c202b3639 service nova] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:56.723526 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580edf32-733a-4c59-939c-2e7c202b3639 service nova] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:56.724761 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580edf32-733a-4c59-939c-2e7c202b3639 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:56.743251 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580edf32-733a-4c59-939c-2e7c202b3639 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:56.743251 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580edf32-733a-4c59-939c-2e7c202b3639 service nova] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:56.755514 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 925/1840] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:10:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:10:56.794385 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56190242-603a-4d60-aff9-a42422a3cfc4 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:56.794385 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56190242-603a-4d60-aff9-a42422a3cfc4 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:56.794385 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56190242-603a-4d60-aff9-a42422a3cfc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:56.875230 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-56190242-603a-4d60-aff9-a42422a3cfc4 None None] MFA Rules not processed for user `2a1319d33e9144859c6010843be312a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:56.883013 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-56190242-603a-4d60-aff9-a42422a3cfc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:56.895361 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580edf32-733a-4c59-939c-2e7c202b3639 service nova] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:56.897049 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-580edf32-733a-4c59-939c-2e7c202b3639 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:56.929844 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 916/1841] 10.222.0.18 () {58 vars in 1018 bytes} [Sat Aug 6 22:10:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:10:56.930438 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580edf32-733a-4c59-939c-2e7c202b3639 service nova] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:56.942335 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a44989-9d9f-4ad7-9f08-72088ae1a449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:56.943231 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a44989-9d9f-4ad7-9f08-72088ae1a449 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:56.943833 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a44989-9d9f-4ad7-9f08-72088ae1a449 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:56.944486 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a44989-9d9f-4ad7-9f08-72088ae1a449 admin admin] PATH_INFO: `/v3/projects/1e947edcc38b45ffa9816a7b3fae80d9` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:56.947208 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a44989-9d9f-4ad7-9f08-72088ae1a449 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e947edcc38b45ffa9816a7b3fae80d9)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:56.968266 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a44989-9d9f-4ad7-9f08-72088ae1a449 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:56.968458 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56190242-603a-4d60-aff9-a42422a3cfc4 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:56.981435 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56190242-603a-4d60-aff9-a42422a3cfc4 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:57.006789 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e20fb3-f353-4cae-a466-55fa756a6635 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a1319d33e9144859c6010843be312a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:57.006789 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e20fb3-f353-4cae-a466-55fa756a6635 service nova] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:57.006789 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e20fb3-f353-4cae-a466-55fa756a6635 service nova] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:57.007339 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e20fb3-f353-4cae-a466-55fa756a6635 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:57.021902 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e20fb3-f353-4cae-a466-55fa756a6635 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:57.023700 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e20fb3-f353-4cae-a466-55fa756a6635 service nova] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:57.037417 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 917/1842] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:10:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 22:10:57.075404 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e20fb3-f353-4cae-a466-55fa756a6635 service nova] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:57.078635 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-e9e20fb3-f353-4cae-a466-55fa756a6635 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:57.111026 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e20fb3-f353-4cae-a466-55fa756a6635 service nova] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:57.116139 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-51a44989-9d9f-4ad7-9f08-72088ae1a449 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e947edcc38b45ffa9816a7b3fae80d9', 'request_id': 'req-51a44989-9d9f-4ad7-9f08-72088ae1a449'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:57.210861 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-51a44989-9d9f-4ad7-9f08-72088ae1a449 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e947edcc38b45ffa9816a7b3fae80d9', 'request_id': 'req-51a44989-9d9f-4ad7-9f08-72088ae1a449'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:57.220517 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-51a44989-9d9f-4ad7-9f08-72088ae1a449 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e947edcc38b45ffa9816a7b3fae80d9', 'request_id': 'req-51a44989-9d9f-4ad7-9f08-72088ae1a449'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:57.253856 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e23f6e8-fcdb-4bcf-a6a9-c14f389cefd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:57.255027 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e23f6e8-fcdb-4bcf-a6a9-c14f389cefd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:57.255027 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e23f6e8-fcdb-4bcf-a6a9-c14f389cefd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:57.255317 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e23f6e8-fcdb-4bcf-a6a9-c14f389cefd9 admin admin] PATH_INFO: `/v3/users/d20eabf28f0b40d3ae1e6f2b3b6d6b8c` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:57.256614 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e23f6e8-fcdb-4bcf-a6a9-c14f389cefd9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d20eabf28f0b40d3ae1e6f2b3b6d6b8c)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:57.259598 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e23f6e8-fcdb-4bcf-a6a9-c14f389cefd9 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:57.260088 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-51a44989-9d9f-4ad7-9f08-72088ae1a449 admin admin] The token cache is being invalidate because project 1e947edcc38b45ffa9816a7b3fae80d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:10:57.260386 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-51a44989-9d9f-4ad7-9f08-72088ae1a449 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:57.264242 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-51a44989-9d9f-4ad7-9f08-72088ae1a449 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:57.266107 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 926/1843] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:10:56 2022] DELETE /identity/v3/projects/1e947edcc38b45ffa9816a7b3fae80d9 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 22:10:57.266860 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a44989-9d9f-4ad7-9f08-72088ae1a449 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:57.279981 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-51a44989-9d9f-4ad7-9f08-72088ae1a449 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:57.312066 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a44989-9d9f-4ad7-9f08-72088ae1a449 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:57.318113 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6415a4b0-2709-43e8-ba9d-73241f4fb802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:57.318842 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6415a4b0-2709-43e8-ba9d-73241f4fb802 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:57.319220 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6415a4b0-2709-43e8-ba9d-73241f4fb802 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:57.320104 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6415a4b0-2709-43e8-ba9d-73241f4fb802 admin admin] PATH_INFO: `/v3/projects/a2bea300a56e4e3a86080740b5c2daa7` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:57.321324 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6415a4b0-2709-43e8-ba9d-73241f4fb802 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2bea300a56e4e3a86080740b5c2daa7)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:57.323223 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6415a4b0-2709-43e8-ba9d-73241f4fb802 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:57.511976 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-0e23f6e8-fcdb-4bcf-a6a9-c14f389cefd9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd20eabf28f0b40d3ae1e6f2b3b6d6b8c'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:57.512176 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-6415a4b0-2709-43e8-ba9d-73241f4fb802 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2bea300a56e4e3a86080740b5c2daa7', 'request_id': 'req-6415a4b0-2709-43e8-ba9d-73241f4fb802'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:57.524209 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-0e23f6e8-fcdb-4bcf-a6a9-c14f389cefd9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd20eabf28f0b40d3ae1e6f2b3b6d6b8c'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:57.531592 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-0e23f6e8-fcdb-4bcf-a6a9-c14f389cefd9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd20eabf28f0b40d3ae1e6f2b3b6d6b8c'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:57.532308 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-0e23f6e8-fcdb-4bcf-a6a9-c14f389cefd9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd20eabf28f0b40d3ae1e6f2b3b6d6b8c'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:57.532952 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-0e23f6e8-fcdb-4bcf-a6a9-c14f389cefd9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd20eabf28f0b40d3ae1e6f2b3b6d6b8c'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:57.595443 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-6415a4b0-2709-43e8-ba9d-73241f4fb802 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2bea300a56e4e3a86080740b5c2daa7', 'request_id': 'req-6415a4b0-2709-43e8-ba9d-73241f4fb802'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:57.601967 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-6415a4b0-2709-43e8-ba9d-73241f4fb802 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2bea300a56e4e3a86080740b5c2daa7', 'request_id': 'req-6415a4b0-2709-43e8-ba9d-73241f4fb802'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:57.613551 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-6415a4b0-2709-43e8-ba9d-73241f4fb802 admin admin] The token cache is being invalidate because project a2bea300a56e4e3a86080740b5c2daa7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:10:57.613841 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-6415a4b0-2709-43e8-ba9d-73241f4fb802 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:57.614505 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-6415a4b0-2709-43e8-ba9d-73241f4fb802 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:57.616024 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 927/1844] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:10:57 2022] DELETE /identity/v3/projects/a2bea300a56e4e3a86080740b5c2daa7 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:10:57.617288 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fed5bc-f826-461b-8935-cf486d4de3f0 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:57.617574 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fed5bc-f826-461b-8935-cf486d4de3f0 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:57.617874 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fed5bc-f826-461b-8935-cf486d4de3f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:57.624361 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:10:57.637653 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-75fed5bc-f826-461b-8935-cf486d4de3f0 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:57.639526 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-75fed5bc-f826-461b-8935-cf486d4de3f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:57.697508 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 928/1845] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:10:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:10:57.698537 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fed5bc-f826-461b-8935-cf486d4de3f0 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:57.698646 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 918/1846] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:10:57 2022] DELETE /identity/v3/users/d20eabf28f0b40d3ae1e6f2b3b6d6b8c => generated 0 bytes in 624 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 22:10:57.703139 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2468d537-ede4-4bc6-9d6f-6f1a301b3d38 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:57.703417 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2468d537-ede4-4bc6-9d6f-6f1a301b3d38 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:57.703692 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2468d537-ede4-4bc6-9d6f-6f1a301b3d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:57.705459 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-75fed5bc-f826-461b-8935-cf486d4de3f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:57.717552 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fed5bc-f826-461b-8935-cf486d4de3f0 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:57.722159 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-2468d537-ede4-4bc6-9d6f-6f1a301b3d38 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:57.723338 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-2468d537-ede4-4bc6-9d6f-6f1a301b3d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:57.775052 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fadf02-4ba1-4b1a-8c48-5e94a0714866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:57.775688 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fadf02-4ba1-4b1a-8c48-5e94a0714866 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:57.775967 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fadf02-4ba1-4b1a-8c48-5e94a0714866 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:57.776237 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fadf02-4ba1-4b1a-8c48-5e94a0714866 admin admin] PATH_INFO: `/v3/users/b6d1bef15904498ea8f678e510d441e0` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:57.777406 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fadf02-4ba1-4b1a-8c48-5e94a0714866 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6d1bef15904498ea8f678e510d441e0)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:57.779305 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fadf02-4ba1-4b1a-8c48-5e94a0714866 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:57.782117 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 919/1847] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:10:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 22:10:57.783358 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9fe769-8999-4935-9c1a-e4e6b33f8eef None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:57.783644 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9fe769-8999-4935-9c1a-e4e6b33f8eef None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:57.785957 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9fe769-8999-4935-9c1a-e4e6b33f8eef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:57.803744 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-1e9fe769-8999-4935-9c1a-e4e6b33f8eef None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:57.804978 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-1e9fe769-8999-4935-9c1a-e4e6b33f8eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:57.814094 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 920/1848] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:10:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:10:57.817564 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9fe769-8999-4935-9c1a-e4e6b33f8eef None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:57.817564 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-1e9fe769-8999-4935-9c1a-e4e6b33f8eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:57.870205 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9fe769-8999-4935-9c1a-e4e6b33f8eef None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:57.874612 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df25cec7-2d88-4f40-977a-c9684bdab98c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:57.875209 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df25cec7-2d88-4f40-977a-c9684bdab98c service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:57.875485 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df25cec7-2d88-4f40-977a-c9684bdab98c service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:57.875753 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df25cec7-2d88-4f40-977a-c9684bdab98c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:57.884651 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df25cec7-2d88-4f40-977a-c9684bdab98c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:57.886171 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df25cec7-2d88-4f40-977a-c9684bdab98c service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:57.896221 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 921/1849] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:10:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:10:57.897039 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df25cec7-2d88-4f40-977a-c9684bdab98c service neutron] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:57.906051 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df25cec7-2d88-4f40-977a-c9684bdab98c service neutron] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:57.910695 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e66f8f-bb8b-4f9f-a72a-1b956a31cb31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:57.911282 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e66f8f-bb8b-4f9f-a72a-1b956a31cb31 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:57.911555 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e66f8f-bb8b-4f9f-a72a-1b956a31cb31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:57.911827 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e66f8f-bb8b-4f9f-a72a-1b956a31cb31 admin admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:57.912525 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e66f8f-bb8b-4f9f-a72a-1b956a31cb31 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 22:10:57.912832 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e66f8f-bb8b-4f9f-a72a-1b956a31cb31 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:57.914398 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e66f8f-bb8b-4f9f-a72a-1b956a31cb31 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:57.926207 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 922/1850] 10.222.0.18 () {58 vars in 1153 bytes} [Sat Aug 6 22:10:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 22:10:57.932578 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e66f8f-bb8b-4f9f-a72a-1b956a31cb31 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:57.940745 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e66f8f-bb8b-4f9f-a72a-1b956a31cb31 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:57.945289 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c9e6a4-bdbd-444c-8c5f-e44ac2bafeca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:57.945875 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c9e6a4-bdbd-444c-8c5f-e44ac2bafeca admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:57.946166 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c9e6a4-bdbd-444c-8c5f-e44ac2bafeca admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:57.946548 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c9e6a4-bdbd-444c-8c5f-e44ac2bafeca admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:57.947017 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c9e6a4-bdbd-444c-8c5f-e44ac2bafeca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:57.948932 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c9e6a4-bdbd-444c-8c5f-e44ac2bafeca admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:58.043449 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-99fadf02-4ba1-4b1a-8c48-5e94a0714866 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6d1bef15904498ea8f678e510d441e0'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:58.055358 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-99fadf02-4ba1-4b1a-8c48-5e94a0714866 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6d1bef15904498ea8f678e510d441e0'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:58.074361 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 923/1851] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:10:57 2022] POST /identity/v3/projects => generated 363 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 22:10:58.074361 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c9e6a4-bdbd-444c-8c5f-e44ac2bafeca admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:58.151746 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c9e6a4-bdbd-444c-8c5f-e44ac2bafeca admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:58.156749 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e062259-83fe-4396-94ef-636a8ac86351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:58.157433 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e062259-83fe-4396-94ef-636a8ac86351 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:58.157799 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e062259-83fe-4396-94ef-636a8ac86351 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:58.158168 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e062259-83fe-4396-94ef-636a8ac86351 admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:58.158859 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e062259-83fe-4396-94ef-636a8ac86351 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:58.160915 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e062259-83fe-4396-94ef-636a8ac86351 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:58.178609 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-99fadf02-4ba1-4b1a-8c48-5e94a0714866 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6d1bef15904498ea8f678e510d441e0'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:58.187352 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-99fadf02-4ba1-4b1a-8c48-5e94a0714866 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6d1bef15904498ea8f678e510d441e0'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:58.189451 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-99fadf02-4ba1-4b1a-8c48-5e94a0714866 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6d1bef15904498ea8f678e510d441e0'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:58.196055 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 929/1852] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:10:57 2022] DELETE /identity/v3/users/b6d1bef15904498ea8f678e510d441e0 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:10:58.293303 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 924/1853] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:10:58 2022] POST /identity/v3/users => generated 282 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 22:10:58.297560 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fadf02-4ba1-4b1a-8c48-5e94a0714866 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:58.298914 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-99fadf02-4ba1-4b1a-8c48-5e94a0714866 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:58.363155 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fadf02-4ba1-4b1a-8c48-5e94a0714866 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:58.372817 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6325cee0-9358-4b4d-adf7-7d49fafc51ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:58.372817 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6325cee0-9358-4b4d-adf7-7d49fafc51ff admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:58.372817 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6325cee0-9358-4b4d-adf7-7d49fafc51ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:58.372817 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6325cee0-9358-4b4d-adf7-7d49fafc51ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:58.373239 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6325cee0-9358-4b4d-adf7-7d49fafc51ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:58.373340 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e062259-83fe-4396-94ef-636a8ac86351 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:58.373471 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6325cee0-9358-4b4d-adf7-7d49fafc51ff admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:58.376256 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:10:58.380209 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-5e062259-83fe-4396-94ef-636a8ac86351 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:58.386104 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 930/1854] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:10:58 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:10:58.390471 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6325cee0-9358-4b4d-adf7-7d49fafc51ff admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:58.407756 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6325cee0-9358-4b4d-adf7-7d49fafc51ff admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:58.411715 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e062259-83fe-4396-94ef-636a8ac86351 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:58.418627 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc4e286-cda5-43b2-9bf3-30ddb8331b0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:58.419260 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc4e286-cda5-43b2-9bf3-30ddb8331b0e admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:58.419603 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc4e286-cda5-43b2-9bf3-30ddb8331b0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:58.419875 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc4e286-cda5-43b2-9bf3-30ddb8331b0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:58.420562 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc4e286-cda5-43b2-9bf3-30ddb8331b0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:58.422169 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc4e286-cda5-43b2-9bf3-30ddb8331b0e admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:58.426272 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06683f4c-7b53-469b-b216-bec833659155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:58.426912 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06683f4c-7b53-469b-b216-bec833659155 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:58.427189 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06683f4c-7b53-469b-b216-bec833659155 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:58.427460 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06683f4c-7b53-469b-b216-bec833659155 admin admin] PATH_INFO: `/v3/projects/d85d0d1786b546ffa511708f4634df21` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:58.428792 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06683f4c-7b53-469b-b216-bec833659155 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d85d0d1786b546ffa511708f4634df21)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:58.437237 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06683f4c-7b53-469b-b216-bec833659155 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:58.441705 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 931/1855] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:10:58 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 22:10:58.445428 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc4e286-cda5-43b2-9bf3-30ddb8331b0e admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:58.457390 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc4e286-cda5-43b2-9bf3-30ddb8331b0e admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:58.462904 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d78088-ff9c-4329-9860-810a5fcbb64a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:58.463021 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d78088-ff9c-4329-9860-810a5fcbb64a admin admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:58.463303 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d78088-ff9c-4329-9860-810a5fcbb64a admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:58.463925 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d78088-ff9c-4329-9860-810a5fcbb64a admin admin] PATH_INFO: `/v3/projects/82c8133fb930408caf9fa6f10d489cc4/users/e9db5f5d5a774b8eb5e121bc46fb466f/roles/56e4a96094e9434cbbafb22579c034f5` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:58.483523 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d78088-ff9c-4329-9860-810a5fcbb64a admin admin] RBAC: Authorizing `identity:create_grant(project_id=82c8133fb930408caf9fa6f10d489cc4, user_id=e9db5f5d5a774b8eb5e121bc46fb466f, role_id=56e4a96094e9434cbbafb22579c034f5)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:58.486614 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d78088-ff9c-4329-9860-810a5fcbb64a admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:58.591566 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 932/1856] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:10:58 2022] PUT /identity/v3/projects/82c8133fb930408caf9fa6f10d489cc4/users/e9db5f5d5a774b8eb5e121bc46fb466f/roles/56e4a96094e9434cbbafb22579c034f5 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:10:58.603210 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd029a6-bdd0-4963-92f3-773ffe42b41d None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:58.603550 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd029a6-bdd0-4963-92f3-773ffe42b41d None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:58.603668 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd029a6-bdd0-4963-92f3-773ffe42b41d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:58.621504 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-06683f4c-7b53-469b-b216-bec833659155 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd85d0d1786b546ffa511708f4634df21', 'request_id': 'req-06683f4c-7b53-469b-b216-bec833659155'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:58.658703 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-afd029a6-bdd0-4963-92f3-773ffe42b41d None None] MFA Rules not processed for user `e9db5f5d5a774b8eb5e121bc46fb466f`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:58.659923 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-afd029a6-bdd0-4963-92f3-773ffe42b41d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:58.687711 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-06683f4c-7b53-469b-b216-bec833659155 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd85d0d1786b546ffa511708f4634df21', 'request_id': 'req-06683f4c-7b53-469b-b216-bec833659155'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:58.699605 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-06683f4c-7b53-469b-b216-bec833659155 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd85d0d1786b546ffa511708f4634df21', 'request_id': 'req-06683f4c-7b53-469b-b216-bec833659155'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:58.710303 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-06683f4c-7b53-469b-b216-bec833659155 admin admin] The token cache is being invalidate because project d85d0d1786b546ffa511708f4634df21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:10:58.710669 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-06683f4c-7b53-469b-b216-bec833659155 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:58.711484 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-06683f4c-7b53-469b-b216-bec833659155 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:58.713114 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 925/1857] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:10:58 2022] DELETE /identity/v3/projects/d85d0d1786b546ffa511708f4634df21 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 22:10:58.718553 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06683f4c-7b53-469b-b216-bec833659155 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:58.720855 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-06683f4c-7b53-469b-b216-bec833659155 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:58.743528 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 933/1858] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:10:58 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:10:58.748157 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd029a6-bdd0-4963-92f3-773ffe42b41d None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:58.750112 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-afd029a6-bdd0-4963-92f3-773ffe42b41d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:58.795467 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06683f4c-7b53-469b-b216-bec833659155 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:58.800573 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939e180f-75e3-4a2d-9d1c-e775776d8253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:58.801197 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939e180f-75e3-4a2d-9d1c-e775776d8253 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:58.801650 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939e180f-75e3-4a2d-9d1c-e775776d8253 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:58.802303 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939e180f-75e3-4a2d-9d1c-e775776d8253 admin admin] PATH_INFO: `/v3/users/8e5caf223bf04e4590f6c226b50113a4` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:58.803621 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939e180f-75e3-4a2d-9d1c-e775776d8253 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e5caf223bf04e4590f6c226b50113a4)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:58.805560 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939e180f-75e3-4a2d-9d1c-e775776d8253 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:58.819995 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd029a6-bdd0-4963-92f3-773ffe42b41d None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:58.824764 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581bdea2-1f9e-4840-8057-0b2996ecb62d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:58.825367 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581bdea2-1f9e-4840-8057-0b2996ecb62d admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:58.826287 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581bdea2-1f9e-4840-8057-0b2996ecb62d admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:58.827021 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581bdea2-1f9e-4840-8057-0b2996ecb62d admin admin] PATH_INFO: `/v3/projects` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:58.827358 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581bdea2-1f9e-4840-8057-0b2996ecb62d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:58.829322 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581bdea2-1f9e-4840-8057-0b2996ecb62d admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:58.945375 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 934/1859] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:10:58 2022] POST /identity/v3/projects => generated 365 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 22:10:58.946346 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581bdea2-1f9e-4840-8057-0b2996ecb62d admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:58.948028 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-581bdea2-1f9e-4840-8057-0b2996ecb62d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:59.012086 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581bdea2-1f9e-4840-8057-0b2996ecb62d admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:59.016783 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cf1807-b410-4f71-bdd2-37b36b043f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:59.017415 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cf1807-b410-4f71-bdd2-37b36b043f09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:59.017693 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cf1807-b410-4f71-bdd2-37b36b043f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:59.017963 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cf1807-b410-4f71-bdd2-37b36b043f09 admin admin] PATH_INFO: `/v3/projects/42fddb3a3b3444e9bf84bf38ec12168e` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:59.019095 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cf1807-b410-4f71-bdd2-37b36b043f09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42fddb3a3b3444e9bf84bf38ec12168e)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:59.021048 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cf1807-b410-4f71-bdd2-37b36b043f09 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:59.026755 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-939e180f-75e3-4a2d-9d1c-e775776d8253 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e5caf223bf04e4590f6c226b50113a4'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:59.038401 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-939e180f-75e3-4a2d-9d1c-e775776d8253 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e5caf223bf04e4590f6c226b50113a4'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:59.045949 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-939e180f-75e3-4a2d-9d1c-e775776d8253 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e5caf223bf04e4590f6c226b50113a4'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:59.046648 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-939e180f-75e3-4a2d-9d1c-e775776d8253 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e5caf223bf04e4590f6c226b50113a4'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:59.047266 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-939e180f-75e3-4a2d-9d1c-e775776d8253 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e5caf223bf04e4590f6c226b50113a4'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:59.132656 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-41cf1807-b410-4f71-bdd2-37b36b043f09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42fddb3a3b3444e9bf84bf38ec12168e', 'request_id': 'req-41cf1807-b410-4f71-bdd2-37b36b043f09'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:59.149553 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 926/1860] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:10:58 2022] DELETE /identity/v3/users/8e5caf223bf04e4590f6c226b50113a4 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 22:10:59.150368 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939e180f-75e3-4a2d-9d1c-e775776d8253 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:59.151694 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-939e180f-75e3-4a2d-9d1c-e775776d8253 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:59.214460 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939e180f-75e3-4a2d-9d1c-e775776d8253 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:59.220869 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec4aa2f-46a7-498e-89a8-f438f6e77500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:59.221485 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec4aa2f-46a7-498e-89a8-f438f6e77500 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:59.221768 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec4aa2f-46a7-498e-89a8-f438f6e77500 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:59.222039 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec4aa2f-46a7-498e-89a8-f438f6e77500 admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:59.222668 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec4aa2f-46a7-498e-89a8-f438f6e77500 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:59.224611 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec4aa2f-46a7-498e-89a8-f438f6e77500 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:59.259072 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-41cf1807-b410-4f71-bdd2-37b36b043f09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42fddb3a3b3444e9bf84bf38ec12168e', 'request_id': 'req-41cf1807-b410-4f71-bdd2-37b36b043f09'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:59.267079 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-41cf1807-b410-4f71-bdd2-37b36b043f09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42fddb3a3b3444e9bf84bf38ec12168e', 'request_id': 'req-41cf1807-b410-4f71-bdd2-37b36b043f09'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:59.279815 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-41cf1807-b410-4f71-bdd2-37b36b043f09 admin admin] The token cache is being invalidate because project 42fddb3a3b3444e9bf84bf38ec12168e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:10:59.280244 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-41cf1807-b410-4f71-bdd2-37b36b043f09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:59.281110 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-41cf1807-b410-4f71-bdd2-37b36b043f09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:59.287023 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 935/1861] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:10:58 2022] DELETE /identity/v3/projects/42fddb3a3b3444e9bf84bf38ec12168e => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:10:59.288141 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ba54f2-a570-4520-b4f5-cee23d8b31f2 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:59.288460 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ba54f2-a570-4520-b4f5-cee23d8b31f2 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:59.288740 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ba54f2-a570-4520-b4f5-cee23d8b31f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:59.308999 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-77ba54f2-a570-4520-b4f5-cee23d8b31f2 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:59.320268 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-77ba54f2-a570-4520-b4f5-cee23d8b31f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:59.320268 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 936/1862] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:10:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 22:10:59.327054 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ce60ec-347c-4f90-bf14-85f2dda128e9 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:59.327336 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ce60ec-347c-4f90-bf14-85f2dda128e9 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:59.327606 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ce60ec-347c-4f90-bf14-85f2dda128e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:59.345955 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 927/1863] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:10:59 2022] POST /identity/v3/users => generated 289 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:10:59.347574 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-87ce60ec-347c-4f90-bf14-85f2dda128e9 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:59.348659 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-87ce60ec-347c-4f90-bf14-85f2dda128e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:59.349937 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec4aa2f-46a7-498e-89a8-f438f6e77500 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:59.351953 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-6ec4aa2f-46a7-498e-89a8-f438f6e77500 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:59.365214 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 937/1864] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:10:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 22:10:59.368386 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec4aa2f-46a7-498e-89a8-f438f6e77500 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:59.369471 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ce60ec-347c-4f90-bf14-85f2dda128e9 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:59.374487 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c04225-371d-4898-9e4f-e1e3f96dc2e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:59.375201 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c04225-371d-4898-9e4f-e1e3f96dc2e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:59.375519 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c04225-371d-4898-9e4f-e1e3f96dc2e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:59.375794 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c04225-371d-4898-9e4f-e1e3f96dc2e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:59.376489 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c04225-371d-4898-9e4f-e1e3f96dc2e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:59.378960 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c04225-371d-4898-9e4f-e1e3f96dc2e9 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:59.380619 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ce60ec-347c-4f90-bf14-85f2dda128e9 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:59.385798 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eca88c7-9b71-4fe1-99ad-089dd8141300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:59.386460 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eca88c7-9b71-4fe1-99ad-089dd8141300 admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:59.386837 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eca88c7-9b71-4fe1-99ad-089dd8141300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:59.387221 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eca88c7-9b71-4fe1-99ad-089dd8141300 admin admin] PATH_INFO: `/v3/domains` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:59.387971 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eca88c7-9b71-4fe1-99ad-089dd8141300 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 22:10:59.389049 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eca88c7-9b71-4fe1-99ad-089dd8141300 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:59.390686 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eca88c7-9b71-4fe1-99ad-089dd8141300 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:59.394755 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 928/1865] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:10:59 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 22:10:59.398466 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c04225-371d-4898-9e4f-e1e3f96dc2e9 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:59.407222 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 938/1866] 10.222.0.18 () {58 vars in 1153 bytes} [Sat Aug 6 22:10:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 22:10:59.413533 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eca88c7-9b71-4fe1-99ad-089dd8141300 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:59.424861 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c04225-371d-4898-9e4f-e1e3f96dc2e9 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:59.429744 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eca88c7-9b71-4fe1-99ad-089dd8141300 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:59.436349 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72812632-001a-4cd7-a06f-f75168ba24f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:59.437057 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72812632-001a-4cd7-a06f-f75168ba24f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:59.437451 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72812632-001a-4cd7-a06f-f75168ba24f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:59.437593 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72812632-001a-4cd7-a06f-f75168ba24f4 admin admin] PATH_INFO: `/v3/projects/ed7e98a4193146df8f32455ecd6c362f/users/09a4f22d75e5494fb014116491a76aaf/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:59.443741 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7a4ca8-f88e-444b-b68b-877f45ef967d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:59.444363 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7a4ca8-f88e-444b-b68b-877f45ef967d admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:59.444668 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7a4ca8-f88e-444b-b68b-877f45ef967d admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:59.444934 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7a4ca8-f88e-444b-b68b-877f45ef967d admin admin] PATH_INFO: `/v3/projects` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:59.445575 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7a4ca8-f88e-444b-b68b-877f45ef967d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:59.460155 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7a4ca8-f88e-444b-b68b-877f45ef967d admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:59.461130 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72812632-001a-4cd7-a06f-f75168ba24f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed7e98a4193146df8f32455ecd6c362f, user_id=09a4f22d75e5494fb014116491a76aaf, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:59.464070 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72812632-001a-4cd7-a06f-f75168ba24f4 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:59.544724 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 929/1867] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:10:59 2022] PUT /identity/v3/projects/ed7e98a4193146df8f32455ecd6c362f/users/09a4f22d75e5494fb014116491a76aaf/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:10:59.548638 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72812632-001a-4cd7-a06f-f75168ba24f4 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:59.582324 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 939/1868] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:10:59 2022] POST /identity/v3/projects => generated 367 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:10:59.587563 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7a4ca8-f88e-444b-b68b-877f45ef967d admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:59.611248 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72812632-001a-4cd7-a06f-f75168ba24f4 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:59.654204 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7a4ca8-f88e-444b-b68b-877f45ef967d admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:59.661028 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16514c5f-bb11-4895-80f7-23f3433d208f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:59.661746 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16514c5f-bb11-4895-80f7-23f3433d208f admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:59.661937 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16514c5f-bb11-4895-80f7-23f3433d208f admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:59.662745 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16514c5f-bb11-4895-80f7-23f3433d208f admin admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:59.672115 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16514c5f-bb11-4895-80f7-23f3433d208f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:59.674756 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16514c5f-bb11-4895-80f7-23f3433d208f admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:59.682604 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e77f932-28c7-4508-a6b9-311d2e535b9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:59.683214 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e77f932-28c7-4508-a6b9-311d2e535b9f admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:59.683520 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e77f932-28c7-4508-a6b9-311d2e535b9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:59.683792 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e77f932-28c7-4508-a6b9-311d2e535b9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:59.684464 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e77f932-28c7-4508-a6b9-311d2e535b9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:59.687169 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e77f932-28c7-4508-a6b9-311d2e535b9f admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:59.700694 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 930/1869] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:10:59 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 22:10:59.702251 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e77f932-28c7-4508-a6b9-311d2e535b9f admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:59.704253 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-7e77f932-28c7-4508-a6b9-311d2e535b9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:59.721867 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e77f932-28c7-4508-a6b9-311d2e535b9f admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:59.728974 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0c0ff8-31a4-4f76-a466-dbc7fb4ba03c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:59.729569 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0c0ff8-31a4-4f76-a466-dbc7fb4ba03c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:59.729855 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0c0ff8-31a4-4f76-a466-dbc7fb4ba03c admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:59.730181 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0c0ff8-31a4-4f76-a466-dbc7fb4ba03c admin admin] PATH_INFO: `/v3/projects/345d71e9fea44171a23fc0b0b93cd2f6` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:59.731588 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0c0ff8-31a4-4f76-a466-dbc7fb4ba03c admin admin] RBAC: Authorizing `identity:delete_project(project_id=345d71e9fea44171a23fc0b0b93cd2f6)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:59.736917 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0c0ff8-31a4-4f76-a466-dbc7fb4ba03c admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:59.769321 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 940/1870] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:10:59 2022] POST /identity/v3/users => generated 284 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 22:10:59.770337 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16514c5f-bb11-4895-80f7-23f3433d208f admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:59.781448 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16514c5f-bb11-4895-80f7-23f3433d208f admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:59.786559 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f999668-b879-4954-80f7-97405fe0d039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:59.787234 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f999668-b879-4954-80f7-97405fe0d039 admin admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:59.787514 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f999668-b879-4954-80f7-97405fe0d039 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:59.787951 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f999668-b879-4954-80f7-97405fe0d039 admin admin] PATH_INFO: `/v3/projects/ed7e98a4193146df8f32455ecd6c362f/users/09a4f22d75e5494fb014116491a76aaf/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:59.790098 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f999668-b879-4954-80f7-97405fe0d039 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed7e98a4193146df8f32455ecd6c362f, user_id=09a4f22d75e5494fb014116491a76aaf, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:59.792818 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f999668-b879-4954-80f7-97405fe0d039 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:59.804532 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 941/1871] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:10:59 2022] PUT /identity/v3/projects/ed7e98a4193146df8f32455ecd6c362f/users/09a4f22d75e5494fb014116491a76aaf/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:10:59.805646 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f999668-b879-4954-80f7-97405fe0d039 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:59.876441 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f999668-b879-4954-80f7-97405fe0d039 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:10:59.890447 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-ac0c0ff8-31a4-4f76-a466-dbc7fb4ba03c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '345d71e9fea44171a23fc0b0b93cd2f6', 'request_id': 'req-ac0c0ff8-31a4-4f76-a466-dbc7fb4ba03c'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:59.940007 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d136b8fb-faeb-4c6c-ad9d-a7409d5433df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:10:59.940661 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d136b8fb-faeb-4c6c-ad9d-a7409d5433df admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:59.940969 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d136b8fb-faeb-4c6c-ad9d-a7409d5433df admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:59.941243 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d136b8fb-faeb-4c6c-ad9d-a7409d5433df admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:59.941890 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d136b8fb-faeb-4c6c-ad9d-a7409d5433df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:10:59.943542 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d136b8fb-faeb-4c6c-ad9d-a7409d5433df admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:10:59.954322 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 942/1872] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:10:59 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:10:59.955633 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f29351-23d5-47d0-9e7c-c3836d3bdc6d None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:10:59.955914 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f29351-23d5-47d0-9e7c-c3836d3bdc6d None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:10:59.956185 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f29351-23d5-47d0-9e7c-c3836d3bdc6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:10:59.972880 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-ac0c0ff8-31a4-4f76-a466-dbc7fb4ba03c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '345d71e9fea44171a23fc0b0b93cd2f6', 'request_id': 'req-ac0c0ff8-31a4-4f76-a466-dbc7fb4ba03c'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:59.979440 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-ac0c0ff8-31a4-4f76-a466-dbc7fb4ba03c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '345d71e9fea44171a23fc0b0b93cd2f6', 'request_id': 'req-ac0c0ff8-31a4-4f76-a466-dbc7fb4ba03c'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:59.983121 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-f9f29351-23d5-47d0-9e7c-c3836d3bdc6d None None] MFA Rules not processed for user `09a4f22d75e5494fb014116491a76aaf`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:10:59.984336 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-f9f29351-23d5-47d0-9e7c-c3836d3bdc6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:10:59.991202 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-ac0c0ff8-31a4-4f76-a466-dbc7fb4ba03c admin admin] The token cache is being invalidate because project 345d71e9fea44171a23fc0b0b93cd2f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:10:59.991486 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-ac0c0ff8-31a4-4f76-a466-dbc7fb4ba03c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:59.992143 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-ac0c0ff8-31a4-4f76-a466-dbc7fb4ba03c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:10:59.993700 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 931/1873] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:10:59 2022] DELETE /identity/v3/projects/345d71e9fea44171a23fc0b0b93cd2f6 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:10:59.994448 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0c0ff8-31a4-4f76-a466-dbc7fb4ba03c admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:10:59.995692 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-ac0c0ff8-31a4-4f76-a466-dbc7fb4ba03c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:00.008881 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0c0ff8-31a4-4f76-a466-dbc7fb4ba03c admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:00.014455 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ee1de2-d814-4e5b-bf39-88a154a48255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:00.015055 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ee1de2-d814-4e5b-bf39-88a154a48255 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:00.015333 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ee1de2-d814-4e5b-bf39-88a154a48255 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:00.015742 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ee1de2-d814-4e5b-bf39-88a154a48255 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:00.016384 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ee1de2-d814-4e5b-bf39-88a154a48255 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:00.018020 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ee1de2-d814-4e5b-bf39-88a154a48255 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:00.027894 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 932/1874] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:10:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:11:00.029131 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ba1ff4-3195-4784-93ab-a841edb1c51c None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:00.029413 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ba1ff4-3195-4784-93ab-a841edb1c51c None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:00.029695 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ba1ff4-3195-4784-93ab-a841edb1c51c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:00.048573 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-17ba1ff4-3195-4784-93ab-a841edb1c51c None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:11:00.049804 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-17ba1ff4-3195-4784-93ab-a841edb1c51c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:00.057683 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 933/1875] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:11:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:11:00.057989 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 943/1876] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:10:59 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:11:00.058920 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ba1ff4-3195-4784-93ab-a841edb1c51c None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:00.067826 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078e8c10-78ed-4bd6-841e-84423bf3ef71 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:00.068112 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078e8c10-78ed-4bd6-841e-84423bf3ef71 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:00.068388 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078e8c10-78ed-4bd6-841e-84423bf3ef71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:00.069965 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ba1ff4-3195-4784-93ab-a841edb1c51c None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:00.074800 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e62996f-72a5-4eb5-89f7-af71695bea72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:00.075616 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e62996f-72a5-4eb5-89f7-af71695bea72 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:00.076521 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e62996f-72a5-4eb5-89f7-af71695bea72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:00.076865 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e62996f-72a5-4eb5-89f7-af71695bea72 admin admin] PATH_INFO: `/v3/projects/1379403844b2452ba2252913f90a6d8d/users/f7c9848c541042f2a5a320638429fb4a/roles/56e4a96094e9434cbbafb22579c034f5` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:00.088769 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-078e8c10-78ed-4bd6-841e-84423bf3ef71 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:11:00.089966 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-078e8c10-78ed-4bd6-841e-84423bf3ef71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:00.091386 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e62996f-72a5-4eb5-89f7-af71695bea72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1379403844b2452ba2252913f90a6d8d, user_id=f7c9848c541042f2a5a320638429fb4a, role_id=56e4a96094e9434cbbafb22579c034f5)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:00.094068 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e62996f-72a5-4eb5-89f7-af71695bea72 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:00.098525 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 944/1877] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:11:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 22:11:00.099362 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078e8c10-78ed-4bd6-841e-84423bf3ef71 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:00.100599 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-078e8c10-78ed-4bd6-841e-84423bf3ef71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:00.104976 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:00.156884 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078e8c10-78ed-4bd6-841e-84423bf3ef71 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:00.160489 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625dc6fe-2039-4aec-a326-b8c9979621c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa71c71de263478da234f3c5716a6e5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:00.161100 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625dc6fe-2039-4aec-a326-b8c9979621c1 service cinder] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:00.161380 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625dc6fe-2039-4aec-a326-b8c9979621c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:00.161652 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625dc6fe-2039-4aec-a326-b8c9979621c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:00.171106 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625dc6fe-2039-4aec-a326-b8c9979621c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:00.172567 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625dc6fe-2039-4aec-a326-b8c9979621c1 service cinder] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:00.174343 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 934/1878] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:11:00 2022] PUT /identity/v3/projects/1379403844b2452ba2252913f90a6d8d/users/f7c9848c541042f2a5a320638429fb4a/roles/56e4a96094e9434cbbafb22579c034f5 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:11:00.182326 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 945/1879] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:11:00 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:11:00.185950 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e62996f-72a5-4eb5-89f7-af71695bea72 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:00.186146 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b88bbaf-66b1-425e-b948-166aab4106a3 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:00.186622 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b88bbaf-66b1-425e-b948-166aab4106a3 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:00.186919 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b88bbaf-66b1-425e-b948-166aab4106a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:00.216433 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-1b88bbaf-66b1-425e-b948-166aab4106a3 None None] MFA Rules not processed for user `f7c9848c541042f2a5a320638429fb4a`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:11:00.217846 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-1b88bbaf-66b1-425e-b948-166aab4106a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:00.249252 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e62996f-72a5-4eb5-89f7-af71695bea72 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:00.254129 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd17d2b-92ce-4dc6-aa4a-f734479a2f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:00.254729 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd17d2b-92ce-4dc6-aa4a-f734479a2f89 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:00.255007 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd17d2b-92ce-4dc6-aa4a-f734479a2f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:00.255279 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd17d2b-92ce-4dc6-aa4a-f734479a2f89 admin admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:00.255921 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd17d2b-92ce-4dc6-aa4a-f734479a2f89 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 22:11:00.256226 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd17d2b-92ce-4dc6-aa4a-f734479a2f89 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:00.257852 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd17d2b-92ce-4dc6-aa4a-f734479a2f89 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:00.276254 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 935/1880] 10.222.0.18 () {58 vars in 1153 bytes} [Sat Aug 6 22:11:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 22:11:00.281386 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd17d2b-92ce-4dc6-aa4a-f734479a2f89 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:00.283036 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 946/1881] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:11:00 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 22:11:00.287401 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b88bbaf-66b1-425e-b948-166aab4106a3 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:00.292138 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd17d2b-92ce-4dc6-aa4a-f734479a2f89 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:00.297763 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7996ae-6d81-4211-9fa2-a3ac834f4b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:00.298427 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7996ae-6d81-4211-9fa2-a3ac834f4b53 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:00.298763 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7996ae-6d81-4211-9fa2-a3ac834f4b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:00.298965 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7996ae-6d81-4211-9fa2-a3ac834f4b53 admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:00.299590 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7996ae-6d81-4211-9fa2-a3ac834f4b53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:00.299865 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b88bbaf-66b1-425e-b948-166aab4106a3 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:00.305610 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7996ae-6d81-4211-9fa2-a3ac834f4b53 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:00.306326 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd2b23c-10c9-4afa-933a-8e16e231e7e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:00.307142 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd2b23c-10c9-4afa-933a-8e16e231e7e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:00.307495 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd2b23c-10c9-4afa-933a-8e16e231e7e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:00.307861 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd2b23c-10c9-4afa-933a-8e16e231e7e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:00.308623 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd2b23c-10c9-4afa-933a-8e16e231e7e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:00.311056 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd2b23c-10c9-4afa-933a-8e16e231e7e3 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:00.385401 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 936/1882] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:11:00 2022] POST /identity/v3/projects => generated 367 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 22:11:00.390261 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7996ae-6d81-4211-9fa2-a3ac834f4b53 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:00.443379 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 947/1883] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:11:00 2022] POST /identity/v3/projects => generated 367 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 22:11:00.447406 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd2b23c-10c9-4afa-933a-8e16e231e7e3 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:00.457305 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7996ae-6d81-4211-9fa2-a3ac834f4b53 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:00.493863 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:00.511163 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd2b23c-10c9-4afa-933a-8e16e231e7e3 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:00.516803 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbb325f-6f2d-4970-a70e-4f5eaa0cc0d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:00.517442 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbb325f-6f2d-4970-a70e-4f5eaa0cc0d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:00.517748 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbb325f-6f2d-4970-a70e-4f5eaa0cc0d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:00.517987 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbb325f-6f2d-4970-a70e-4f5eaa0cc0d1 admin admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:00.518623 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbb325f-6f2d-4970-a70e-4f5eaa0cc0d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:00.520570 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbb325f-6f2d-4970-a70e-4f5eaa0cc0d1 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:00.524672 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac60f3f-18df-4f78-8524-7054e4fc51a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:00.524672 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac60f3f-18df-4f78-8524-7054e4fc51a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:00.524672 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac60f3f-18df-4f78-8524-7054e4fc51a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:00.524672 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac60f3f-18df-4f78-8524-7054e4fc51a6 admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:00.525076 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac60f3f-18df-4f78-8524-7054e4fc51a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:00.526235 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac60f3f-18df-4f78-8524-7054e4fc51a6 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:00.624618 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 948/1884] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:11:00 2022] POST /identity/v3/users => generated 290 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 22:11:00.625380 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbb325f-6f2d-4970-a70e-4f5eaa0cc0d1 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:00.629083 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-0bbb325f-6f2d-4970-a70e-4f5eaa0cc0d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:00.664700 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 937/1885] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:11:00 2022] POST /identity/v3/users => generated 284 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 22:11:00.665817 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac60f3f-18df-4f78-8524-7054e4fc51a6 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:00.676169 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac60f3f-18df-4f78-8524-7054e4fc51a6 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:00.680703 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43defc6d-6b52-4e5d-8b57-f13fa4f41de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:00.681285 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43defc6d-6b52-4e5d-8b57-f13fa4f41de4 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:00.681558 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43defc6d-6b52-4e5d-8b57-f13fa4f41de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:00.681824 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43defc6d-6b52-4e5d-8b57-f13fa4f41de4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:00.682459 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43defc6d-6b52-4e5d-8b57-f13fa4f41de4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:00.684059 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43defc6d-6b52-4e5d-8b57-f13fa4f41de4 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:00.687266 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbb325f-6f2d-4970-a70e-4f5eaa0cc0d1 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:00.691113 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e8b801-2ad3-4087-8053-8b4fc3b1343d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa71c71de263478da234f3c5716a6e5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:00.691773 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e8b801-2ad3-4087-8053-8b4fc3b1343d service cinder] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:00.691992 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e8b801-2ad3-4087-8053-8b4fc3b1343d service cinder] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:00.692264 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e8b801-2ad3-4087-8053-8b4fc3b1343d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:00.694057 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-82e8b801-2ad3-4087-8053-8b4fc3b1343d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:00.694708 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 938/1886] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:11:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:11:00.695512 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43defc6d-6b52-4e5d-8b57-f13fa4f41de4 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:00.704850 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43defc6d-6b52-4e5d-8b57-f13fa4f41de4 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:00.709501 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44c7585-42ac-405c-9e00-028f86fb97a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:00.710080 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44c7585-42ac-405c-9e00-028f86fb97a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:00.710358 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44c7585-42ac-405c-9e00-028f86fb97a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:00.710627 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44c7585-42ac-405c-9e00-028f86fb97a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:00.711245 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44c7585-42ac-405c-9e00-028f86fb97a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:00.713049 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44c7585-42ac-405c-9e00-028f86fb97a7 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:00.724771 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 939/1887] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:11:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:11:00.725711 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44c7585-42ac-405c-9e00-028f86fb97a7 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:00.735640 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44c7585-42ac-405c-9e00-028f86fb97a7 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:00.740294 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e84b9c-3870-4cd2-9b95-b2f405b09a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:00.740973 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e84b9c-3870-4cd2-9b95-b2f405b09a61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:00.741251 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e84b9c-3870-4cd2-9b95-b2f405b09a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:00.741523 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e84b9c-3870-4cd2-9b95-b2f405b09a61 admin admin] PATH_INFO: `/v3/projects/508eae5d8f214f31b68fced6a56140d8/users/c8fa8f6e617a46869563267c2451fb17/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:00.756803 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e84b9c-3870-4cd2-9b95-b2f405b09a61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=508eae5d8f214f31b68fced6a56140d8, user_id=c8fa8f6e617a46869563267c2451fb17, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:00.758248 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e8b801-2ad3-4087-8053-8b4fc3b1343d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:00.759448 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e84b9c-3870-4cd2-9b95-b2f405b09a61 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:00.759667 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e8b801-2ad3-4087-8053-8b4fc3b1343d service cinder] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:00.769850 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 949/1888] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:11:00 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:11:00.770672 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e8b801-2ad3-4087-8053-8b4fc3b1343d service cinder] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:00.780907 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e8b801-2ad3-4087-8053-8b4fc3b1343d service cinder] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:00.786533 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113b5d2e-b3a6-4270-97bc-91b1ed35c08e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:00.787180 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113b5d2e-b3a6-4270-97bc-91b1ed35c08e admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:00.787409 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113b5d2e-b3a6-4270-97bc-91b1ed35c08e admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:00.787684 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113b5d2e-b3a6-4270-97bc-91b1ed35c08e admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:00.788327 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113b5d2e-b3a6-4270-97bc-91b1ed35c08e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:00.789950 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113b5d2e-b3a6-4270-97bc-91b1ed35c08e admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:00.804616 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 950/1889] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:11:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:11:00.809111 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113b5d2e-b3a6-4270-97bc-91b1ed35c08e admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:00.820195 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113b5d2e-b3a6-4270-97bc-91b1ed35c08e admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:00.825632 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c40f7f8-f8c8-4e74-ba13-58b5042ff261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:00.826464 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c40f7f8-f8c8-4e74-ba13-58b5042ff261 admin admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:00.826748 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c40f7f8-f8c8-4e74-ba13-58b5042ff261 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:00.827025 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c40f7f8-f8c8-4e74-ba13-58b5042ff261 admin admin] PATH_INFO: `/v3/projects/db58e9425d9a47a897a549253129b6f8/users/252d554a7bdb4e5784862bb76c78de67/roles/56e4a96094e9434cbbafb22579c034f5` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:00.842924 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c40f7f8-f8c8-4e74-ba13-58b5042ff261 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db58e9425d9a47a897a549253129b6f8, user_id=252d554a7bdb4e5784862bb76c78de67, role_id=56e4a96094e9434cbbafb22579c034f5)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:00.845825 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c40f7f8-f8c8-4e74-ba13-58b5042ff261 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:00.876002 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 940/1890] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:11:00 2022] PUT /identity/v3/projects/508eae5d8f214f31b68fced6a56140d8/users/c8fa8f6e617a46869563267c2451fb17/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:11:00.879828 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e84b9c-3870-4cd2-9b95-b2f405b09a61 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:00.943255 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e84b9c-3870-4cd2-9b95-b2f405b09a61 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:00.948698 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199c5433-876b-4452-9624-9e7351330095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:00.949392 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199c5433-876b-4452-9624-9e7351330095 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:00.949745 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199c5433-876b-4452-9624-9e7351330095 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:00.950095 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199c5433-876b-4452-9624-9e7351330095 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:00.951109 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199c5433-876b-4452-9624-9e7351330095 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:00.952968 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199c5433-876b-4452-9624-9e7351330095 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:00.964204 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 941/1891] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:11:00 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 22:11:00.968704 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199c5433-876b-4452-9624-9e7351330095 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:00.973010 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 951/1892] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:11:00 2022] PUT /identity/v3/projects/db58e9425d9a47a897a549253129b6f8/users/252d554a7bdb4e5784862bb76c78de67/roles/56e4a96094e9434cbbafb22579c034f5 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:11:00.983739 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199c5433-876b-4452-9624-9e7351330095 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:00.984168 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca1a398-48d1-4121-b9f2-c772d66af77f None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:00.984668 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca1a398-48d1-4121-b9f2-c772d66af77f None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:00.985108 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca1a398-48d1-4121-b9f2-c772d66af77f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:01.025612 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-fca1a398-48d1-4121-b9f2-c772d66af77f None None] MFA Rules not processed for user `252d554a7bdb4e5784862bb76c78de67`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:11:01.027091 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-fca1a398-48d1-4121-b9f2-c772d66af77f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:01.054938 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ab5bda-355b-413c-b807-f1b9784f92cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:01.056120 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab5bda-355b-413c-b807-f1b9784f92cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:01.056598 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab5bda-355b-413c-b807-f1b9784f92cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:01.057018 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab5bda-355b-413c-b807-f1b9784f92cc admin admin] PATH_INFO: `/v3/projects/508eae5d8f214f31b68fced6a56140d8/users/c8fa8f6e617a46869563267c2451fb17/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:01.059248 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ab5bda-355b-413c-b807-f1b9784f92cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=508eae5d8f214f31b68fced6a56140d8, user_id=c8fa8f6e617a46869563267c2451fb17, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:01.062686 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ab5bda-355b-413c-b807-f1b9784f92cc admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:01.074261 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 942/1893] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:11:00 2022] PUT /identity/v3/projects/508eae5d8f214f31b68fced6a56140d8/users/c8fa8f6e617a46869563267c2451fb17/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 22:11:01.084543 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5081ebd-1ea2-45c2-9870-f3f7794c5048 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:01.084831 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5081ebd-1ea2-45c2-9870-f3f7794c5048 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:01.085158 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5081ebd-1ea2-45c2-9870-f3f7794c5048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:01.102702 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 952/1894] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:11:00 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:11:01.107477 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca1a398-48d1-4121-b9f2-c772d66af77f None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:01.118341 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca1a398-48d1-4121-b9f2-c772d66af77f None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:01.118581 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-f5081ebd-1ea2-45c2-9870-f3f7794c5048 None None] MFA Rules not processed for user `c8fa8f6e617a46869563267c2451fb17`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:11:01.120040 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-f5081ebd-1ea2-45c2-9870-f3f7794c5048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:01.195644 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 943/1895] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:11:01 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 22:11:01.201253 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4252b888-ce53-401e-8a49-818adf1922ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:01.201918 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4252b888-ce53-401e-8a49-818adf1922ba admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:01.202205 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4252b888-ce53-401e-8a49-818adf1922ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:01.202483 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4252b888-ce53-401e-8a49-818adf1922ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:01.209627 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5081ebd-1ea2-45c2-9870-f3f7794c5048 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:01.210956 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4252b888-ce53-401e-8a49-818adf1922ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:01.219820 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4252b888-ce53-401e-8a49-818adf1922ba admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:01.277858 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5081ebd-1ea2-45c2-9870-f3f7794c5048 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:01.281900 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddacfbe-c0c7-4b42-9acb-4aaadc37afc7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa71c71de263478da234f3c5716a6e5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:01.282545 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddacfbe-c0c7-4b42-9acb-4aaadc37afc7 service cinder] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:01.282825 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddacfbe-c0c7-4b42-9acb-4aaadc37afc7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:01.283095 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddacfbe-c0c7-4b42-9acb-4aaadc37afc7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:01.293468 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddacfbe-c0c7-4b42-9acb-4aaadc37afc7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:01.295626 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddacfbe-c0c7-4b42-9acb-4aaadc37afc7 service cinder] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:01.296211 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 953/1896] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:11:01 2022] POST /identity/v3/projects => generated 367 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 22:11:01.300232 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4252b888-ce53-401e-8a49-818adf1922ba admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:01.309187 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 944/1897] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:11:01 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:11:01.319435 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:01.396466 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4252b888-ce53-401e-8a49-818adf1922ba admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:01.401258 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4724a04a-d6cf-4fd1-83e6-deb02e668c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:01.401859 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4724a04a-d6cf-4fd1-83e6-deb02e668c43 admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:01.402136 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4724a04a-d6cf-4fd1-83e6-deb02e668c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:01.402407 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4724a04a-d6cf-4fd1-83e6-deb02e668c43 admin admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:01.403029 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4724a04a-d6cf-4fd1-83e6-deb02e668c43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:01.405151 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4724a04a-d6cf-4fd1-83e6-deb02e668c43 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:01.500601 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:01.506206 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 954/1898] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:11:01 2022] POST /identity/v3/users => generated 290 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 22:11:01.510166 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddacfbe-c0c7-4b42-9acb-4aaadc37afc7 service cinder] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:01.521186 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddacfbe-c0c7-4b42-9acb-4aaadc37afc7 service cinder] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:01.525771 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67851e7a-7440-4754-b424-0e030c92a270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:01.526374 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67851e7a-7440-4754-b424-0e030c92a270 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:01.526652 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67851e7a-7440-4754-b424-0e030c92a270 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:01.526952 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67851e7a-7440-4754-b424-0e030c92a270 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:01.527623 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67851e7a-7440-4754-b424-0e030c92a270 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:01.529766 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67851e7a-7440-4754-b424-0e030c92a270 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:01.540871 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 945/1899] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:11:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:11:01.546063 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4724a04a-d6cf-4fd1-83e6-deb02e668c43 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:01.556243 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4724a04a-d6cf-4fd1-83e6-deb02e668c43 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:01.560968 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d232fd5-0bac-4185-962f-cd419d9f3529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:01.561625 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d232fd5-0bac-4185-962f-cd419d9f3529 admin admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:01.561902 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d232fd5-0bac-4185-962f-cd419d9f3529 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:01.562173 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d232fd5-0bac-4185-962f-cd419d9f3529 admin admin] PATH_INFO: `/v3/projects/010734f77a6b438592b89cbd8852ae6d/users/56abfbcb63074d7aa05d263eee1a275e/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:01.577040 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d232fd5-0bac-4185-962f-cd419d9f3529 admin admin] RBAC: Authorizing `identity:create_grant(project_id=010734f77a6b438592b89cbd8852ae6d, user_id=56abfbcb63074d7aa05d263eee1a275e, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:01.579678 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d232fd5-0bac-4185-962f-cd419d9f3529 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:01.601896 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67851e7a-7440-4754-b424-0e030c92a270 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:01.672687 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67851e7a-7440-4754-b424-0e030c92a270 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:01.677049 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2d9938-c221-4c5c-8c03-65b1e4f76ee8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa71c71de263478da234f3c5716a6e5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:01.677673 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2d9938-c221-4c5c-8c03-65b1e4f76ee8 service cinder] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:01.677954 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2d9938-c221-4c5c-8c03-65b1e4f76ee8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:01.678310 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2d9938-c221-4c5c-8c03-65b1e4f76ee8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:01.734447 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 955/1900] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:11:01 2022] PUT /identity/v3/projects/010734f77a6b438592b89cbd8852ae6d/users/56abfbcb63074d7aa05d263eee1a275e/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:11:01.738239 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d232fd5-0bac-4185-962f-cd419d9f3529 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:01.766051 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2d9938-c221-4c5c-8c03-65b1e4f76ee8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:01.768654 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2d9938-c221-4c5c-8c03-65b1e4f76ee8 service cinder] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:01.802445 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 946/1901] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:11:01 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 22:11:01.855014 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d232fd5-0bac-4185-962f-cd419d9f3529 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:01.861981 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef1c0d5-b77b-4221-a6a8-6feec5eec06d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:01.862632 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef1c0d5-b77b-4221-a6a8-6feec5eec06d admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:01.865725 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef1c0d5-b77b-4221-a6a8-6feec5eec06d admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:01.865725 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef1c0d5-b77b-4221-a6a8-6feec5eec06d admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:01.866282 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef1c0d5-b77b-4221-a6a8-6feec5eec06d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:01.868441 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef1c0d5-b77b-4221-a6a8-6feec5eec06d admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:01.883984 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 956/1902] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:11:01 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 22:11:01.886482 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2d9938-c221-4c5c-8c03-65b1e4f76ee8 service cinder] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:01.900453 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2d9938-c221-4c5c-8c03-65b1e4f76ee8 service cinder] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:01.906169 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7fc203-d775-4710-a54f-ac796b78967d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:01.906930 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7fc203-d775-4710-a54f-ac796b78967d admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:01.907283 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7fc203-d775-4710-a54f-ac796b78967d admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:01.907562 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7fc203-d775-4710-a54f-ac796b78967d admin admin] PATH_INFO: `/v3/projects/010734f77a6b438592b89cbd8852ae6d/users/56abfbcb63074d7aa05d263eee1a275e/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:01.914670 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7fc203-d775-4710-a54f-ac796b78967d admin admin] RBAC: Authorizing `identity:create_grant(project_id=010734f77a6b438592b89cbd8852ae6d, user_id=56abfbcb63074d7aa05d263eee1a275e, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:01.917897 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7fc203-d775-4710-a54f-ac796b78967d admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:01.960199 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 947/1903] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:11:01 2022] PUT /identity/v3/projects/010734f77a6b438592b89cbd8852ae6d/users/56abfbcb63074d7aa05d263eee1a275e/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:11:01.962261 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddffde01-c0f5-426b-a0a4-8f24e12bed3d None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:01.962975 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddffde01-c0f5-426b-a0a4-8f24e12bed3d None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:01.963344 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddffde01-c0f5-426b-a0a4-8f24e12bed3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:02.000686 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-ddffde01-c0f5-426b-a0a4-8f24e12bed3d None None] MFA Rules not processed for user `56abfbcb63074d7aa05d263eee1a275e`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:11:02.003037 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-ddffde01-c0f5-426b-a0a4-8f24e12bed3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:02.084237 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 957/1904] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:11:01 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:11:02.095301 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7fc203-d775-4710-a54f-ac796b78967d admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:02.158068 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7fc203-d775-4710-a54f-ac796b78967d admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:02.162195 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4ea085-5b91-4e79-b77e-7186d1fa5643 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa71c71de263478da234f3c5716a6e5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:02.162865 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4ea085-5b91-4e79-b77e-7186d1fa5643 service cinder] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:02.163165 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4ea085-5b91-4e79-b77e-7186d1fa5643 service cinder] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:02.163403 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4ea085-5b91-4e79-b77e-7186d1fa5643 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:02.174187 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4ea085-5b91-4e79-b77e-7186d1fa5643 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:02.175611 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4ea085-5b91-4e79-b77e-7186d1fa5643 service cinder] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:02.191867 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 948/1905] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:11:02 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:11:02.367736 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:03.108115 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddffde01-c0f5-426b-a0a4-8f24e12bed3d None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:03.110546 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-ddffde01-c0f5-426b-a0a4-8f24e12bed3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:03.217225 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddffde01-c0f5-426b-a0a4-8f24e12bed3d None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:03.221716 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d096467a-5c88-4cf8-8a8a-4b39724dc76b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:03.222342 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d096467a-5c88-4cf8-8a8a-4b39724dc76b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:03.222663 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d096467a-5c88-4cf8-8a8a-4b39724dc76b admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:03.222934 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d096467a-5c88-4cf8-8a8a-4b39724dc76b admin admin] PATH_INFO: `/v3/users/88b5e81665d84915ab400ebe8f9f1c61` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:03.224052 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d096467a-5c88-4cf8-8a8a-4b39724dc76b admin admin] RBAC: Authorizing `identity:delete_user(user_id=88b5e81665d84915ab400ebe8f9f1c61)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:03.226474 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d096467a-5c88-4cf8-8a8a-4b39724dc76b admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:03.416835 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:03.489737 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-d096467a-5c88-4cf8-8a8a-4b39724dc76b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88b5e81665d84915ab400ebe8f9f1c61'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:03.513597 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-d096467a-5c88-4cf8-8a8a-4b39724dc76b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88b5e81665d84915ab400ebe8f9f1c61'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:03.653122 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-d096467a-5c88-4cf8-8a8a-4b39724dc76b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88b5e81665d84915ab400ebe8f9f1c61'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:03.653544 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:03.661860 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-d096467a-5c88-4cf8-8a8a-4b39724dc76b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88b5e81665d84915ab400ebe8f9f1c61'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:03.662591 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-d096467a-5c88-4cf8-8a8a-4b39724dc76b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88b5e81665d84915ab400ebe8f9f1c61'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:03.671052 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 958/1906] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:11:03 2022] DELETE /identity/v3/users/88b5e81665d84915ab400ebe8f9f1c61 => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:11:03.675384 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da998ff3-ef29-4b88-b419-272e279b9016 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:03.675669 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da998ff3-ef29-4b88-b419-272e279b9016 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:03.675977 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da998ff3-ef29-4b88-b419-272e279b9016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:03.695800 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-da998ff3-ef29-4b88-b419-272e279b9016 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:11:03.697072 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-da998ff3-ef29-4b88-b419-272e279b9016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:03.766060 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 949/1907] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:11:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:11:03.775601 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d096467a-5c88-4cf8-8a8a-4b39724dc76b admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:03.783028 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-d096467a-5c88-4cf8-8a8a-4b39724dc76b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:03.840448 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d096467a-5c88-4cf8-8a8a-4b39724dc76b admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:03.845185 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f68661-28ab-4fa5-bc53-6a2bcf3b53ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:03.846178 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f68661-28ab-4fa5-bc53-6a2bcf3b53ac service neutron] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:03.846468 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f68661-28ab-4fa5-bc53-6a2bcf3b53ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:03.846752 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f68661-28ab-4fa5-bc53-6a2bcf3b53ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:03.858010 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f68661-28ab-4fa5-bc53-6a2bcf3b53ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:03.859450 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f68661-28ab-4fa5-bc53-6a2bcf3b53ac service neutron] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:03.880639 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 959/1908] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:11:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:11:04.185995 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:04.500599 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da998ff3-ef29-4b88-b419-272e279b9016 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:04.502067 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-da998ff3-ef29-4b88-b419-272e279b9016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:04.517585 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da998ff3-ef29-4b88-b419-272e279b9016 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:04.526257 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a69f437-e693-4d1a-96f5-756c23cb8004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:04.527098 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a69f437-e693-4d1a-96f5-756c23cb8004 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:04.527434 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a69f437-e693-4d1a-96f5-756c23cb8004 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:04.527790 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a69f437-e693-4d1a-96f5-756c23cb8004 admin admin] PATH_INFO: `/v3/projects/540fabe5da6d4928a4411e4c96cb4256` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:04.529263 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a69f437-e693-4d1a-96f5-756c23cb8004 admin admin] RBAC: Authorizing `identity:delete_project(project_id=540fabe5da6d4928a4411e4c96cb4256)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:04.531775 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a69f437-e693-4d1a-96f5-756c23cb8004 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:04.676577 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-5a69f437-e693-4d1a-96f5-756c23cb8004 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '540fabe5da6d4928a4411e4c96cb4256', 'request_id': 'req-5a69f437-e693-4d1a-96f5-756c23cb8004'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:04.750166 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-5a69f437-e693-4d1a-96f5-756c23cb8004 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '540fabe5da6d4928a4411e4c96cb4256', 'request_id': 'req-5a69f437-e693-4d1a-96f5-756c23cb8004'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:04.757610 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-5a69f437-e693-4d1a-96f5-756c23cb8004 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '540fabe5da6d4928a4411e4c96cb4256', 'request_id': 'req-5a69f437-e693-4d1a-96f5-756c23cb8004'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:04.769763 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-5a69f437-e693-4d1a-96f5-756c23cb8004 admin admin] The token cache is being invalidate because project 540fabe5da6d4928a4411e4c96cb4256 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:11:04.770065 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-5a69f437-e693-4d1a-96f5-756c23cb8004 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:04.770735 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-5a69f437-e693-4d1a-96f5-756c23cb8004 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:04.772245 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 950/1909] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:11:04 2022] DELETE /identity/v3/projects/540fabe5da6d4928a4411e4c96cb4256 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:11:04.776396 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f68661-28ab-4fa5-bc53-6a2bcf3b53ac service neutron] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:04.786785 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f68661-28ab-4fa5-bc53-6a2bcf3b53ac service neutron] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:04.845984 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf7952e-6a0b-48a0-ac19-00c7ab620ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:04.846712 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf7952e-6a0b-48a0-ac19-00c7ab620ca2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:04.846993 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf7952e-6a0b-48a0-ac19-00c7ab620ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:04.847264 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf7952e-6a0b-48a0-ac19-00c7ab620ca2 admin admin] PATH_INFO: `/v3/users/94de72fe0e5047cb95ce0ac27d86b23e` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:04.854479 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf7952e-6a0b-48a0-ac19-00c7ab620ca2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94de72fe0e5047cb95ce0ac27d86b23e)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:04.854479 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf7952e-6a0b-48a0-ac19-00c7ab620ca2 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:05.045107 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-8cf7952e-6a0b-48a0-ac19-00c7ab620ca2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94de72fe0e5047cb95ce0ac27d86b23e'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:05.066598 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-8cf7952e-6a0b-48a0-ac19-00c7ab620ca2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94de72fe0e5047cb95ce0ac27d86b23e'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:05.139294 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-8cf7952e-6a0b-48a0-ac19-00c7ab620ca2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94de72fe0e5047cb95ce0ac27d86b23e'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:05.148925 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-8cf7952e-6a0b-48a0-ac19-00c7ab620ca2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94de72fe0e5047cb95ce0ac27d86b23e'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:05.149923 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-8cf7952e-6a0b-48a0-ac19-00c7ab620ca2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94de72fe0e5047cb95ce0ac27d86b23e'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:05.158312 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 960/1910] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:11:04 2022] DELETE /identity/v3/users/94de72fe0e5047cb95ce0ac27d86b23e => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:11:05.190306 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:05.510518 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:06.082315 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a69f437-e693-4d1a-96f5-756c23cb8004 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:06.083776 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-5a69f437-e693-4d1a-96f5-756c23cb8004 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:06.096612 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a69f437-e693-4d1a-96f5-756c23cb8004 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:06.156208 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e9cf90-224e-45fb-aa53-702feb9f4f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:06.156880 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e9cf90-224e-45fb-aa53-702feb9f4f0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:06.157159 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e9cf90-224e-45fb-aa53-702feb9f4f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:06.157425 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e9cf90-224e-45fb-aa53-702feb9f4f0e admin admin] PATH_INFO: `/v3/projects/c5db62acdba04a64b5dc8d09add306bc` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:06.158574 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e9cf90-224e-45fb-aa53-702feb9f4f0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5db62acdba04a64b5dc8d09add306bc)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:06.160526 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e9cf90-224e-45fb-aa53-702feb9f4f0e admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:06.334213 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-a0e9cf90-224e-45fb-aa53-702feb9f4f0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5db62acdba04a64b5dc8d09add306bc', 'request_id': 'req-a0e9cf90-224e-45fb-aa53-702feb9f4f0e'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:06.424116 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-a0e9cf90-224e-45fb-aa53-702feb9f4f0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5db62acdba04a64b5dc8d09add306bc', 'request_id': 'req-a0e9cf90-224e-45fb-aa53-702feb9f4f0e'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:06.431422 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-a0e9cf90-224e-45fb-aa53-702feb9f4f0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5db62acdba04a64b5dc8d09add306bc', 'request_id': 'req-a0e9cf90-224e-45fb-aa53-702feb9f4f0e'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:06.443100 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-a0e9cf90-224e-45fb-aa53-702feb9f4f0e admin admin] The token cache is being invalidate because project c5db62acdba04a64b5dc8d09add306bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:11:06.443490 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-a0e9cf90-224e-45fb-aa53-702feb9f4f0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:06.444234 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-a0e9cf90-224e-45fb-aa53-702feb9f4f0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:06.445824 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 951/1911] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:11:06 2022] DELETE /identity/v3/projects/c5db62acdba04a64b5dc8d09add306bc => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:11:06.451219 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adad8211-a231-4d53-b2dd-f13982402eed None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:06.452001 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adad8211-a231-4d53-b2dd-f13982402eed None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:06.452499 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adad8211-a231-4d53-b2dd-f13982402eed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:06.477077 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-adad8211-a231-4d53-b2dd-f13982402eed None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:11:06.479624 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-adad8211-a231-4d53-b2dd-f13982402eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:06.549053 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 961/1912] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:11:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:11:06.557323 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eb7ddc-7dbf-4e83-8033-e4fc26b77f7f None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:06.557615 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eb7ddc-7dbf-4e83-8033-e4fc26b77f7f None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:06.557880 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eb7ddc-7dbf-4e83-8033-e4fc26b77f7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:06.583581 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-42eb7ddc-7dbf-4e83-8033-e4fc26b77f7f None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:11:06.584869 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-42eb7ddc-7dbf-4e83-8033-e4fc26b77f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:06.593142 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 952/1913] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:11:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:11:06.597663 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adad8211-a231-4d53-b2dd-f13982402eed None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:06.608098 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adad8211-a231-4d53-b2dd-f13982402eed None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:06.612974 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4422e16a-cb33-4b15-9bc7-e8d1cfd90a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:06.613576 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4422e16a-cb33-4b15-9bc7-e8d1cfd90a4b admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:06.613852 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4422e16a-cb33-4b15-9bc7-e8d1cfd90a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:06.614121 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4422e16a-cb33-4b15-9bc7-e8d1cfd90a4b admin admin] PATH_INFO: `/v3/domains` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:06.614765 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4422e16a-cb33-4b15-9bc7-e8d1cfd90a4b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 22:11:06.615063 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4422e16a-cb33-4b15-9bc7-e8d1cfd90a4b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:06.616706 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4422e16a-cb33-4b15-9bc7-e8d1cfd90a4b admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:06.634767 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 962/1914] 10.222.0.18 () {58 vars in 1153 bytes} [Sat Aug 6 22:11:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 22:11:06.635631 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42eb7ddc-7dbf-4e83-8033-e4fc26b77f7f None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:06.646229 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42eb7ddc-7dbf-4e83-8033-e4fc26b77f7f None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:06.651244 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef269a2-ea4d-4854-ba01-cff955c11f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:06.651857 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef269a2-ea4d-4854-ba01-cff955c11f4b admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:06.652140 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef269a2-ea4d-4854-ba01-cff955c11f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:06.652431 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef269a2-ea4d-4854-ba01-cff955c11f4b admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:06.653080 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef269a2-ea4d-4854-ba01-cff955c11f4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:06.655501 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef269a2-ea4d-4854-ba01-cff955c11f4b admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:06.795167 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 953/1915] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:11:06 2022] POST /identity/v3/projects => generated 387 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 22:11:06.796343 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4422e16a-cb33-4b15-9bc7-e8d1cfd90a4b admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:06.806483 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4422e16a-cb33-4b15-9bc7-e8d1cfd90a4b admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:06.869278 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c69efb5-3f07-4f62-a027-ea96553914a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:06.871831 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c69efb5-3f07-4f62-a027-ea96553914a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:06.871831 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c69efb5-3f07-4f62-a027-ea96553914a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:06.871831 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c69efb5-3f07-4f62-a027-ea96553914a6 admin admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:06.871831 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c69efb5-3f07-4f62-a027-ea96553914a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:06.873208 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c69efb5-3f07-4f62-a027-ea96553914a6 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:07.037243 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 963/1916] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:11:06 2022] POST /identity/v3/users => generated 294 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:11:07.045846 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef269a2-ea4d-4854-ba01-cff955c11f4b admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:07.057263 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef269a2-ea4d-4854-ba01-cff955c11f4b admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:07.064506 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:07.080051 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d7375c-a117-4beb-a453-821491c3f94c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:07.080640 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c69efb5-3f07-4f62-a027-ea96553914a6 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:07.080763 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d7375c-a117-4beb-a453-821491c3f94c admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:07.080975 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d7375c-a117-4beb-a453-821491c3f94c admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:07.081244 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d7375c-a117-4beb-a453-821491c3f94c admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:07.081890 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d7375c-a117-4beb-a453-821491c3f94c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:07.082362 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-3c69efb5-3f07-4f62-a027-ea96553914a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:07.083482 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d7375c-a117-4beb-a453-821491c3f94c admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:07.097514 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c69efb5-3f07-4f62-a027-ea96553914a6 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:07.102656 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 954/1917] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:11:07 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 22:11:07.104850 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c7dcc3-e1b6-46c8-86cd-e956b6f7f292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:07.104850 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c7dcc3-e1b6-46c8-86cd-e956b6f7f292 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:07.105247 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c7dcc3-e1b6-46c8-86cd-e956b6f7f292 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:07.105683 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c7dcc3-e1b6-46c8-86cd-e956b6f7f292 admin admin] PATH_INFO: `/v3/users/252d554a7bdb4e5784862bb76c78de67` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:07.107456 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c7dcc3-e1b6-46c8-86cd-e956b6f7f292 admin admin] RBAC: Authorizing `identity:delete_user(user_id=252d554a7bdb4e5784862bb76c78de67)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:07.108688 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d7375c-a117-4beb-a453-821491c3f94c admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:07.109699 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c7dcc3-e1b6-46c8-86cd-e956b6f7f292 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:07.126639 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d7375c-a117-4beb-a453-821491c3f94c admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:07.131651 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b167a604-6762-4cb2-a88b-44fd95b13d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:07.134032 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b167a604-6762-4cb2-a88b-44fd95b13d47 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:07.134032 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b167a604-6762-4cb2-a88b-44fd95b13d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:07.134032 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b167a604-6762-4cb2-a88b-44fd95b13d47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:07.134032 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b167a604-6762-4cb2-a88b-44fd95b13d47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:07.135327 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b167a604-6762-4cb2-a88b-44fd95b13d47 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:07.146103 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 955/1918] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:11:07 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:11:07.150392 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b167a604-6762-4cb2-a88b-44fd95b13d47 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:07.160086 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b167a604-6762-4cb2-a88b-44fd95b13d47 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:07.164742 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0a889c-e7d9-4fa7-b468-612075f14f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:07.165418 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0a889c-e7d9-4fa7-b468-612075f14f33 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:07.165701 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0a889c-e7d9-4fa7-b468-612075f14f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:07.165975 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0a889c-e7d9-4fa7-b468-612075f14f33 admin admin] PATH_INFO: `/v3/projects/403a34cd096a4ec58957164cb5d1081f/users/d5c14955d7de45d1aec0e9e573adf5c8/roles/56e4a96094e9434cbbafb22579c034f5` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:07.180732 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0a889c-e7d9-4fa7-b468-612075f14f33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=403a34cd096a4ec58957164cb5d1081f, user_id=d5c14955d7de45d1aec0e9e573adf5c8, role_id=56e4a96094e9434cbbafb22579c034f5)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:07.183437 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0a889c-e7d9-4fa7-b468-612075f14f33 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:07.264463 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 956/1919] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:11:07 2022] PUT /identity/v3/projects/403a34cd096a4ec58957164cb5d1081f/users/d5c14955d7de45d1aec0e9e573adf5c8/roles/56e4a96094e9434cbbafb22579c034f5 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 22:11:07.274420 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acdeb48-7cd8-4c38-b82c-bf81320743cb None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:07.274709 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acdeb48-7cd8-4c38-b82c-bf81320743cb None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:07.274988 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acdeb48-7cd8-4c38-b82c-bf81320743cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:07.292073 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-e1c7dcc3-e1b6-46c8-86cd-e956b6f7f292 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '252d554a7bdb4e5784862bb76c78de67'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:07.304716 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-e1c7dcc3-e1b6-46c8-86cd-e956b6f7f292 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '252d554a7bdb4e5784862bb76c78de67'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:07.316578 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-9acdeb48-7cd8-4c38-b82c-bf81320743cb None None] MFA Rules not processed for user `d5c14955d7de45d1aec0e9e573adf5c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:11:07.316578 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-9acdeb48-7cd8-4c38-b82c-bf81320743cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:07.378079 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-e1c7dcc3-e1b6-46c8-86cd-e956b6f7f292 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '252d554a7bdb4e5784862bb76c78de67'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:07.385510 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-e1c7dcc3-e1b6-46c8-86cd-e956b6f7f292 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '252d554a7bdb4e5784862bb76c78de67'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:07.386146 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-e1c7dcc3-e1b6-46c8-86cd-e956b6f7f292 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '252d554a7bdb4e5784862bb76c78de67'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:07.389644 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 957/1920] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:11:07 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 22:11:07.390927 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:07.392693 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 964/1921] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:11:07 2022] DELETE /identity/v3/users/252d554a7bdb4e5784862bb76c78de67 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:11:07.394034 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acdeb48-7cd8-4c38-b82c-bf81320743cb None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:07.396973 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9430744c-dfd6-47ce-917f-dc889a1cba83 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:07.397260 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9430744c-dfd6-47ce-917f-dc889a1cba83 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:07.397533 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9430744c-dfd6-47ce-917f-dc889a1cba83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:07.416618 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-9430744c-dfd6-47ce-917f-dc889a1cba83 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:11:07.417945 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-9430744c-dfd6-47ce-917f-dc889a1cba83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:07.456294 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acdeb48-7cd8-4c38-b82c-bf81320743cb None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:07.485636 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 965/1922] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:11:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:11:07.498845 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9430744c-dfd6-47ce-917f-dc889a1cba83 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:07.500480 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-9430744c-dfd6-47ce-917f-dc889a1cba83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:07.521904 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6f4e03-bf12-4a44-b28b-cfc23c5e7341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:07.521904 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6f4e03-bf12-4a44-b28b-cfc23c5e7341 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:07.521904 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6f4e03-bf12-4a44-b28b-cfc23c5e7341 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:07.521904 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6f4e03-bf12-4a44-b28b-cfc23c5e7341 admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:07.522815 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6f4e03-bf12-4a44-b28b-cfc23c5e7341 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:07.522815 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6f4e03-bf12-4a44-b28b-cfc23c5e7341 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:07.563024 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9430744c-dfd6-47ce-917f-dc889a1cba83 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:07.566732 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b59c01-f78f-483e-904b-48f5274e18a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:07.567342 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b59c01-f78f-483e-904b-48f5274e18a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:07.567623 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b59c01-f78f-483e-904b-48f5274e18a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:07.567898 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b59c01-f78f-483e-904b-48f5274e18a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:07.577548 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b59c01-f78f-483e-904b-48f5274e18a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:07.578973 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b59c01-f78f-483e-904b-48f5274e18a2 service neutron] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:07.588877 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 966/1923] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:11:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 22:11:07.678900 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 958/1924] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:11:07 2022] POST /identity/v3/projects => generated 389 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:11:07.683352 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b59c01-f78f-483e-904b-48f5274e18a2 service neutron] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:07.688329 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-c3b59c01-f78f-483e-904b-48f5274e18a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:07.705455 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b59c01-f78f-483e-904b-48f5274e18a2 service neutron] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:07.784564 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696b79ba-0103-437c-8440-754930341fcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:07.785308 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b79ba-0103-437c-8440-754930341fcd admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:07.785639 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b79ba-0103-437c-8440-754930341fcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:07.785875 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b79ba-0103-437c-8440-754930341fcd admin admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:07.786472 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696b79ba-0103-437c-8440-754930341fcd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:07.788536 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696b79ba-0103-437c-8440-754930341fcd admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:07.941801 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 967/1925] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:11:07 2022] POST /identity/v3/users => generated 301 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:11:07.952204 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6f4e03-bf12-4a44-b28b-cfc23c5e7341 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:07.966616 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6f4e03-bf12-4a44-b28b-cfc23c5e7341 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:07.974206 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe32412-3907-4d6f-9e44-6408b5bd2d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:07.974872 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe32412-3907-4d6f-9e44-6408b5bd2d9f admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:07.975148 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe32412-3907-4d6f-9e44-6408b5bd2d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:07.976213 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe32412-3907-4d6f-9e44-6408b5bd2d9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:07.976906 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe32412-3907-4d6f-9e44-6408b5bd2d9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:07.978970 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe32412-3907-4d6f-9e44-6408b5bd2d9f admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:07.994792 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 959/1926] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:11:07 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:11:07.996314 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696b79ba-0103-437c-8440-754930341fcd admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:08.007916 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696b79ba-0103-437c-8440-754930341fcd admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:08.014928 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9907b1-62fe-4335-8988-82a6ccf1f721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:08.014928 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9907b1-62fe-4335-8988-82a6ccf1f721 admin admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:08.014928 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9907b1-62fe-4335-8988-82a6ccf1f721 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:08.014928 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9907b1-62fe-4335-8988-82a6ccf1f721 admin admin] PATH_INFO: `/v3/projects/e95bcdc3503c4137ac0c949ab5964a15/users/4a3b0c33b3544d4aa6859ff60a35c991/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:08.032062 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9907b1-62fe-4335-8988-82a6ccf1f721 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e95bcdc3503c4137ac0c949ab5964a15, user_id=4a3b0c33b3544d4aa6859ff60a35c991, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:08.034865 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9907b1-62fe-4335-8988-82a6ccf1f721 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:08.145033 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 968/1927] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:11:07 2022] PUT /identity/v3/projects/e95bcdc3503c4137ac0c949ab5964a15/users/4a3b0c33b3544d4aa6859ff60a35c991/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:11:08.149109 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe32412-3907-4d6f-9e44-6408b5bd2d9f admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:08.159683 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe32412-3907-4d6f-9e44-6408b5bd2d9f admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:08.174744 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9907b1-62fe-4335-8988-82a6ccf1f721 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:08.176500 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-ab9907b1-62fe-4335-8988-82a6ccf1f721 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:08.258860 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ff0026-25d7-4cc5-8e7f-491f2810327c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:08.259903 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ff0026-25d7-4cc5-8e7f-491f2810327c admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:08.260633 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ff0026-25d7-4cc5-8e7f-491f2810327c admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:08.260976 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ff0026-25d7-4cc5-8e7f-491f2810327c admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:08.261814 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ff0026-25d7-4cc5-8e7f-491f2810327c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:08.264156 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ff0026-25d7-4cc5-8e7f-491f2810327c admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:08.274668 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 960/1928] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:11:08 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:11:08.281695 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ff0026-25d7-4cc5-8e7f-491f2810327c admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:08.284917 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9907b1-62fe-4335-8988-82a6ccf1f721 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:08.289783 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5aba5d-7f61-4a02-9786-18a93387ae98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:08.290353 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5aba5d-7f61-4a02-9786-18a93387ae98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:08.290632 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5aba5d-7f61-4a02-9786-18a93387ae98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:08.290924 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5aba5d-7f61-4a02-9786-18a93387ae98 admin admin] PATH_INFO: `/v3/projects/db58e9425d9a47a897a549253129b6f8` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:08.291581 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ff0026-25d7-4cc5-8e7f-491f2810327c admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:08.292241 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5aba5d-7f61-4a02-9786-18a93387ae98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db58e9425d9a47a897a549253129b6f8)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:08.294223 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5aba5d-7f61-4a02-9786-18a93387ae98 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:08.297677 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4d9096-6e85-4643-bc94-427961e2f493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:08.301414 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4d9096-6e85-4643-bc94-427961e2f493 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:08.301414 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4d9096-6e85-4643-bc94-427961e2f493 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:08.301414 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4d9096-6e85-4643-bc94-427961e2f493 admin admin] PATH_INFO: `/v3/projects/e95bcdc3503c4137ac0c949ab5964a15/users/4a3b0c33b3544d4aa6859ff60a35c991/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:08.301414 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4d9096-6e85-4643-bc94-427961e2f493 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e95bcdc3503c4137ac0c949ab5964a15, user_id=4a3b0c33b3544d4aa6859ff60a35c991, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:08.304122 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4d9096-6e85-4643-bc94-427961e2f493 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:08.317706 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 961/1929] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:11:08 2022] PUT /identity/v3/projects/e95bcdc3503c4137ac0c949ab5964a15/users/4a3b0c33b3544d4aa6859ff60a35c991/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:11:08.330798 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc44975d-1e2a-46e0-83c8-32500b48504d None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:08.331090 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc44975d-1e2a-46e0-83c8-32500b48504d None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:08.331387 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc44975d-1e2a-46e0-83c8-32500b48504d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:08.366261 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-fc44975d-1e2a-46e0-83c8-32500b48504d None None] MFA Rules not processed for user `4a3b0c33b3544d4aa6859ff60a35c991`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:11:08.367476 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-fc44975d-1e2a-46e0-83c8-32500b48504d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:08.435078 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-ff5aba5d-7f61-4a02-9786-18a93387ae98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db58e9425d9a47a897a549253129b6f8', 'request_id': 'req-ff5aba5d-7f61-4a02-9786-18a93387ae98'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:08.437218 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 962/1930] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:11:08 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 22:11:08.450303 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc44975d-1e2a-46e0-83c8-32500b48504d None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:08.451894 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-fc44975d-1e2a-46e0-83c8-32500b48504d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:08.503964 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-ff5aba5d-7f61-4a02-9786-18a93387ae98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db58e9425d9a47a897a549253129b6f8', 'request_id': 'req-ff5aba5d-7f61-4a02-9786-18a93387ae98'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:08.510019 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-ff5aba5d-7f61-4a02-9786-18a93387ae98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db58e9425d9a47a897a549253129b6f8', 'request_id': 'req-ff5aba5d-7f61-4a02-9786-18a93387ae98'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:08.512630 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc44975d-1e2a-46e0-83c8-32500b48504d None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:08.521699 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-ff5aba5d-7f61-4a02-9786-18a93387ae98 admin admin] The token cache is being invalidate because project db58e9425d9a47a897a549253129b6f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:11:08.521988 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-ff5aba5d-7f61-4a02-9786-18a93387ae98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:08.522771 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-ff5aba5d-7f61-4a02-9786-18a93387ae98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:08.524475 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 969/1931] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:11:08 2022] DELETE /identity/v3/projects/db58e9425d9a47a897a549253129b6f8 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 22:11:08.529165 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5aba5d-7f61-4a02-9786-18a93387ae98 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:08.530622 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-ff5aba5d-7f61-4a02-9786-18a93387ae98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:08.563525 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1722e34-b4e6-408c-ac26-960e2cf02a7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa71c71de263478da234f3c5716a6e5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:08.564025 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1722e34-b4e6-408c-ac26-960e2cf02a7b service cinder] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:08.564303 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1722e34-b4e6-408c-ac26-960e2cf02a7b service cinder] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:08.564611 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1722e34-b4e6-408c-ac26-960e2cf02a7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:08.566439 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-d1722e34-b4e6-408c-ac26-960e2cf02a7b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:08.597014 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5aba5d-7f61-4a02-9786-18a93387ae98 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:08.607008 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26641271-11eb-4a23-a3c6-c38dcec64213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:08.607920 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26641271-11eb-4a23-a3c6-c38dcec64213 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:08.608204 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26641271-11eb-4a23-a3c6-c38dcec64213 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:08.608506 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26641271-11eb-4a23-a3c6-c38dcec64213 admin admin] PATH_INFO: `/v3/users/56abfbcb63074d7aa05d263eee1a275e` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:08.609698 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26641271-11eb-4a23-a3c6-c38dcec64213 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56abfbcb63074d7aa05d263eee1a275e)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:08.611740 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26641271-11eb-4a23-a3c6-c38dcec64213 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:08.634581 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1722e34-b4e6-408c-ac26-960e2cf02a7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:08.636329 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1722e34-b4e6-408c-ac26-960e2cf02a7b service cinder] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:08.647410 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 963/1932] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:11:08 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 22:11:08.671204 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:08.789433 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1722e34-b4e6-408c-ac26-960e2cf02a7b service cinder] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:08.791301 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-d1722e34-b4e6-408c-ac26-960e2cf02a7b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:08.806217 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1722e34-b4e6-408c-ac26-960e2cf02a7b service cinder] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:08.811259 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784fe5ce-4732-4038-af13-0068cf0b010d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:08.811782 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784fe5ce-4732-4038-af13-0068cf0b010d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:08.812052 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784fe5ce-4732-4038-af13-0068cf0b010d admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:08.812323 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784fe5ce-4732-4038-af13-0068cf0b010d admin admin] PATH_INFO: `/v3/users/d5c14955d7de45d1aec0e9e573adf5c8` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:08.813584 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784fe5ce-4732-4038-af13-0068cf0b010d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5c14955d7de45d1aec0e9e573adf5c8)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:08.815525 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784fe5ce-4732-4038-af13-0068cf0b010d admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:08.942164 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-26641271-11eb-4a23-a3c6-c38dcec64213 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56abfbcb63074d7aa05d263eee1a275e'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:08.956129 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-26641271-11eb-4a23-a3c6-c38dcec64213 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56abfbcb63074d7aa05d263eee1a275e'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:09.063795 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-26641271-11eb-4a23-a3c6-c38dcec64213 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56abfbcb63074d7aa05d263eee1a275e'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:09.071150 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-26641271-11eb-4a23-a3c6-c38dcec64213 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56abfbcb63074d7aa05d263eee1a275e'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:09.071802 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-26641271-11eb-4a23-a3c6-c38dcec64213 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56abfbcb63074d7aa05d263eee1a275e'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:09.078866 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 970/1933] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:11:08 2022] DELETE /identity/v3/users/56abfbcb63074d7aa05d263eee1a275e => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 22:11:09.100470 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:09.259183 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-784fe5ce-4732-4038-af13-0068cf0b010d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5c14955d7de45d1aec0e9e573adf5c8'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:09.275614 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-784fe5ce-4732-4038-af13-0068cf0b010d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5c14955d7de45d1aec0e9e573adf5c8'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:09.285038 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-784fe5ce-4732-4038-af13-0068cf0b010d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5c14955d7de45d1aec0e9e573adf5c8'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:09.285921 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-784fe5ce-4732-4038-af13-0068cf0b010d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5c14955d7de45d1aec0e9e573adf5c8'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:09.286619 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-784fe5ce-4732-4038-af13-0068cf0b010d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5c14955d7de45d1aec0e9e573adf5c8'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:09.401142 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 964/1934] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:11:08 2022] DELETE /identity/v3/users/d5c14955d7de45d1aec0e9e573adf5c8 => generated 0 bytes in 612 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:11:09.409931 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1917b22-3515-486b-8366-d55b34569ab9 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:09.410218 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1917b22-3515-486b-8366-d55b34569ab9 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:09.410495 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1917b22-3515-486b-8366-d55b34569ab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:09.439291 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-b1917b22-3515-486b-8366-d55b34569ab9 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:11:09.440288 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-b1917b22-3515-486b-8366-d55b34569ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:09.504626 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 971/1935] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:11:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:11:09.545604 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784fe5ce-4732-4038-af13-0068cf0b010d admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:09.547151 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-784fe5ce-4732-4038-af13-0068cf0b010d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:09.608250 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:INSERT=4,keystone:SELECT=80,keystone:DELETE=16 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:09.631704 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784fe5ce-4732-4038-af13-0068cf0b010d admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:09.635594 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7e7116-2cfc-410e-8647-d50717c42710 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:09.636151 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7e7116-2cfc-410e-8647-d50717c42710 service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:09.636509 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7e7116-2cfc-410e-8647-d50717c42710 service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:09.636731 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7e7116-2cfc-410e-8647-d50717c42710 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:09.652834 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7e7116-2cfc-410e-8647-d50717c42710 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:09.654917 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7e7116-2cfc-410e-8647-d50717c42710 service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:09.670478 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 965/1936] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:11:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:11:09.674817 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:10.034373 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1917b22-3515-486b-8366-d55b34569ab9 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:10.036232 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-b1917b22-3515-486b-8366-d55b34569ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:10.052123 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1917b22-3515-486b-8366-d55b34569ab9 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:10.058526 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8ce2e3-26db-4719-a531-e28cb327263f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:10.059939 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8ce2e3-26db-4719-a531-e28cb327263f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:10.060371 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8ce2e3-26db-4719-a531-e28cb327263f admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:10.060854 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8ce2e3-26db-4719-a531-e28cb327263f admin admin] PATH_INFO: `/v3/projects/010734f77a6b438592b89cbd8852ae6d` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:10.062229 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8ce2e3-26db-4719-a531-e28cb327263f admin admin] RBAC: Authorizing `identity:delete_project(project_id=010734f77a6b438592b89cbd8852ae6d)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:10.065243 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8ce2e3-26db-4719-a531-e28cb327263f admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:10.293594 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-dc8ce2e3-26db-4719-a531-e28cb327263f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '010734f77a6b438592b89cbd8852ae6d', 'request_id': 'req-dc8ce2e3-26db-4719-a531-e28cb327263f'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:10.412752 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-dc8ce2e3-26db-4719-a531-e28cb327263f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '010734f77a6b438592b89cbd8852ae6d', 'request_id': 'req-dc8ce2e3-26db-4719-a531-e28cb327263f'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:10.419875 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-dc8ce2e3-26db-4719-a531-e28cb327263f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '010734f77a6b438592b89cbd8852ae6d', 'request_id': 'req-dc8ce2e3-26db-4719-a531-e28cb327263f'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:10.436883 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-dc8ce2e3-26db-4719-a531-e28cb327263f admin admin] The token cache is being invalidate because project 010734f77a6b438592b89cbd8852ae6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:11:10.448910 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-dc8ce2e3-26db-4719-a531-e28cb327263f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:10.451159 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-dc8ce2e3-26db-4719-a531-e28cb327263f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:10.454895 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 972/1937] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:11:10 2022] DELETE /identity/v3/projects/010734f77a6b438592b89cbd8852ae6d => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:11:10.459784 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426e4be6-eba2-4e2b-bacd-32808b44d2c6 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:10.460262 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426e4be6-eba2-4e2b-bacd-32808b44d2c6 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:10.460489 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426e4be6-eba2-4e2b-bacd-32808b44d2c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:10.482651 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-426e4be6-eba2-4e2b-bacd-32808b44d2c6 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:11:10.483993 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-426e4be6-eba2-4e2b-bacd-32808b44d2c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:10.545446 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 966/1938] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:11:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:11:10.553235 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efd7b33-008a-4e67-8e38-e4067c8b8e02 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:10.553736 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efd7b33-008a-4e67-8e38-e4067c8b8e02 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:10.553926 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efd7b33-008a-4e67-8e38-e4067c8b8e02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:10.572995 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-8efd7b33-008a-4e67-8e38-e4067c8b8e02 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:11:10.574182 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-8efd7b33-008a-4e67-8e38-e4067c8b8e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:10.582102 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 973/1939] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:11:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:11:10.586482 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426e4be6-eba2-4e2b-bacd-32808b44d2c6 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:10.596758 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426e4be6-eba2-4e2b-bacd-32808b44d2c6 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:10.602182 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb5076e-2f28-4510-b4bb-b24bc59da855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:10.604522 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb5076e-2f28-4510-b4bb-b24bc59da855 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:10.604522 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb5076e-2f28-4510-b4bb-b24bc59da855 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:10.604522 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb5076e-2f28-4510-b4bb-b24bc59da855 admin admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:10.604522 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb5076e-2f28-4510-b4bb-b24bc59da855 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 22:11:10.604522 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb5076e-2f28-4510-b4bb-b24bc59da855 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:10.605819 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb5076e-2f28-4510-b4bb-b24bc59da855 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:10.618376 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 967/1940] 10.222.0.18 () {58 vars in 1153 bytes} [Sat Aug 6 22:11:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 22:11:10.623164 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efd7b33-008a-4e67-8e38-e4067c8b8e02 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:10.636790 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efd7b33-008a-4e67-8e38-e4067c8b8e02 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:10.637934 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb5076e-2f28-4510-b4bb-b24bc59da855 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:10.645641 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-bbb5076e-2f28-4510-b4bb-b24bc59da855 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:10.655983 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee8908c-fddb-433b-ac36-b79b3bba56d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:10.656683 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee8908c-fddb-433b-ac36-b79b3bba56d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:10.656957 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee8908c-fddb-433b-ac36-b79b3bba56d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:10.657222 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee8908c-fddb-433b-ac36-b79b3bba56d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:10.658029 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee8908c-fddb-433b-ac36-b79b3bba56d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:10.660182 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee8908c-fddb-433b-ac36-b79b3bba56d1 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:10.660806 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb5076e-2f28-4510-b4bb-b24bc59da855 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:10.667875 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bd1212-7c34-4eda-b354-26a8963888ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:10.667875 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd1212-7c34-4eda-b354-26a8963888ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:10.668208 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd1212-7c34-4eda-b354-26a8963888ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:10.669599 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd1212-7c34-4eda-b354-26a8963888ef admin admin] PATH_INFO: `/v3/projects/403a34cd096a4ec58957164cb5d1081f` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:10.671039 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bd1212-7c34-4eda-b354-26a8963888ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=403a34cd096a4ec58957164cb5d1081f)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:10.673047 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bd1212-7c34-4eda-b354-26a8963888ef admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:10.717168 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:DELETE=1,keystone:SELECT=95,keystone:INSERT=4 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:10.751728 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 974/1941] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:11:10 2022] POST /identity/v3/projects => generated 387 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 22:11:10.755829 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee8908c-fddb-433b-ac36-b79b3bba56d1 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:10.803022 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-33bd1212-7c34-4eda-b354-26a8963888ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '403a34cd096a4ec58957164cb5d1081f', 'request_id': 'req-33bd1212-7c34-4eda-b354-26a8963888ef'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:10.819702 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee8908c-fddb-433b-ac36-b79b3bba56d1 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:10.882132 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560174eb-d8ba-4457-88c7-19074c65a378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:10.882741 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560174eb-d8ba-4457-88c7-19074c65a378 admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:10.883018 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560174eb-d8ba-4457-88c7-19074c65a378 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:10.883315 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560174eb-d8ba-4457-88c7-19074c65a378 admin admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:10.883937 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560174eb-d8ba-4457-88c7-19074c65a378 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:10.885856 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560174eb-d8ba-4457-88c7-19074c65a378 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:10.932235 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-33bd1212-7c34-4eda-b354-26a8963888ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '403a34cd096a4ec58957164cb5d1081f', 'request_id': 'req-33bd1212-7c34-4eda-b354-26a8963888ef'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:10.938378 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-33bd1212-7c34-4eda-b354-26a8963888ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '403a34cd096a4ec58957164cb5d1081f', 'request_id': 'req-33bd1212-7c34-4eda-b354-26a8963888ef'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:10.951127 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-33bd1212-7c34-4eda-b354-26a8963888ef admin admin] The token cache is being invalidate because project 403a34cd096a4ec58957164cb5d1081f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:11:10.951456 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-33bd1212-7c34-4eda-b354-26a8963888ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:10.952120 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-33bd1212-7c34-4eda-b354-26a8963888ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:10.953680 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 968/1942] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:11:10 2022] DELETE /identity/v3/projects/403a34cd096a4ec58957164cb5d1081f => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:11:10.957722 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bd1212-7c34-4eda-b354-26a8963888ef admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:10.959134 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-33bd1212-7c34-4eda-b354-26a8963888ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:10.972396 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bd1212-7c34-4eda-b354-26a8963888ef admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:10.977538 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec0f2d1-86a4-4e51-8d6f-b47c43fa42a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:10.978146 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0f2d1-86a4-4e51-8d6f-b47c43fa42a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:10.978448 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0f2d1-86a4-4e51-8d6f-b47c43fa42a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:10.978722 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0f2d1-86a4-4e51-8d6f-b47c43fa42a1 admin admin] PATH_INFO: `/v3/users/4a3b0c33b3544d4aa6859ff60a35c991` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:10.979936 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec0f2d1-86a4-4e51-8d6f-b47c43fa42a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a3b0c33b3544d4aa6859ff60a35c991)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:10.982286 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec0f2d1-86a4-4e51-8d6f-b47c43fa42a1 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:10.989136 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:11.035652 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 975/1943] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:11:10 2022] POST /identity/v3/users => generated 294 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 22:11:11.039604 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560174eb-d8ba-4457-88c7-19074c65a378 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:11.041206 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-560174eb-d8ba-4457-88c7-19074c65a378 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:11.053707 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560174eb-d8ba-4457-88c7-19074c65a378 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:11.058412 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1633ec2e-30d1-4c48-aa2c-3d6e036113df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:11.058993 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1633ec2e-30d1-4c48-aa2c-3d6e036113df admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:11.059343 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1633ec2e-30d1-4c48-aa2c-3d6e036113df admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:11.060429 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1633ec2e-30d1-4c48-aa2c-3d6e036113df admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:11.061256 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1633ec2e-30d1-4c48-aa2c-3d6e036113df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:11.062839 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1633ec2e-30d1-4c48-aa2c-3d6e036113df admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:11.072711 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 976/1944] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:11:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 22:11:11.076567 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1633ec2e-30d1-4c48-aa2c-3d6e036113df admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:11.085772 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1633ec2e-30d1-4c48-aa2c-3d6e036113df admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:11.090268 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc22460d-e3f5-4c15-b81d-232b3f54fd44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:11.091017 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc22460d-e3f5-4c15-b81d-232b3f54fd44 admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:11.091293 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc22460d-e3f5-4c15-b81d-232b3f54fd44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:11.091563 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc22460d-e3f5-4c15-b81d-232b3f54fd44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:11.092173 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc22460d-e3f5-4c15-b81d-232b3f54fd44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:11.093729 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc22460d-e3f5-4c15-b81d-232b3f54fd44 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:11.104122 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 977/1945] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:11:11 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 22:11:11.108547 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc22460d-e3f5-4c15-b81d-232b3f54fd44 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:11.117655 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc22460d-e3f5-4c15-b81d-232b3f54fd44 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:11.122871 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122566eb-4147-4c0a-8e3b-eee36fc8634c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:11.123466 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122566eb-4147-4c0a-8e3b-eee36fc8634c admin admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:11.127765 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122566eb-4147-4c0a-8e3b-eee36fc8634c admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:11.127765 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122566eb-4147-4c0a-8e3b-eee36fc8634c admin admin] PATH_INFO: `/v3/projects/b54123196a8f46ae9298e89e4262c2d2/users/97c467309b774af99980909e5a94841f/roles/56e4a96094e9434cbbafb22579c034f5` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:11.137644 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122566eb-4147-4c0a-8e3b-eee36fc8634c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b54123196a8f46ae9298e89e4262c2d2, user_id=97c467309b774af99980909e5a94841f, role_id=56e4a96094e9434cbbafb22579c034f5)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:11.140241 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122566eb-4147-4c0a-8e3b-eee36fc8634c admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:11.230090 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 978/1946] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:11:11 2022] PUT /identity/v3/projects/b54123196a8f46ae9298e89e4262c2d2/users/97c467309b774af99980909e5a94841f/roles/56e4a96094e9434cbbafb22579c034f5 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 22:11:11.241188 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4768c762-55ef-4e62-953c-f0f96b286dc9 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:11.245142 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4768c762-55ef-4e62-953c-f0f96b286dc9 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:11.245142 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4768c762-55ef-4e62-953c-f0f96b286dc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:11.270325 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-4768c762-55ef-4e62-953c-f0f96b286dc9 None None] MFA Rules not processed for user `97c467309b774af99980909e5a94841f`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:11:11.271681 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-4768c762-55ef-4e62-953c-f0f96b286dc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:11.281536 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-bec0f2d1-86a4-4e51-8d6f-b47c43fa42a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a3b0c33b3544d4aa6859ff60a35c991'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:11.295692 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-bec0f2d1-86a4-4e51-8d6f-b47c43fa42a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a3b0c33b3544d4aa6859ff60a35c991'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:11.303058 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-bec0f2d1-86a4-4e51-8d6f-b47c43fa42a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a3b0c33b3544d4aa6859ff60a35c991'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:11.303811 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-bec0f2d1-86a4-4e51-8d6f-b47c43fa42a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a3b0c33b3544d4aa6859ff60a35c991'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:11.304476 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-bec0f2d1-86a4-4e51-8d6f-b47c43fa42a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a3b0c33b3544d4aa6859ff60a35c991'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:11.362286 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 979/1947] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:11:11 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:11:11.370803 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4768c762-55ef-4e62-953c-f0f96b286dc9 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:11.375519 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-4768c762-55ef-4e62-953c-f0f96b286dc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:11.395206 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 969/1948] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:11:10 2022] DELETE /identity/v3/users/4a3b0c33b3544d4aa6859ff60a35c991 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 22:11:11.448791 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4768c762-55ef-4e62-953c-f0f96b286dc9 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:11.531089 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155e4f18-2be3-40be-af25-0fbf387baf54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:11.531702 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155e4f18-2be3-40be-af25-0fbf387baf54 admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:11.531980 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155e4f18-2be3-40be-af25-0fbf387baf54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:11.532251 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155e4f18-2be3-40be-af25-0fbf387baf54 admin admin] PATH_INFO: `/v3/projects` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:11.532917 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155e4f18-2be3-40be-af25-0fbf387baf54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:11.534800 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155e4f18-2be3-40be-af25-0fbf387baf54 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:11.625864 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 980/1949] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:11:11 2022] POST /identity/v3/projects => generated 387 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 22:11:11.630236 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec0f2d1-86a4-4e51-8d6f-b47c43fa42a1 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:11.653380 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec0f2d1-86a4-4e51-8d6f-b47c43fa42a1 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:11.713056 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e6e283-7e7e-4bdc-bf85-2159e0478aab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:11.713668 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e6e283-7e7e-4bdc-bf85-2159e0478aab admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:11.713953 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e6e283-7e7e-4bdc-bf85-2159e0478aab admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:11.714224 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e6e283-7e7e-4bdc-bf85-2159e0478aab admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:11.714860 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e6e283-7e7e-4bdc-bf85-2159e0478aab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:11.716946 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e6e283-7e7e-4bdc-bf85-2159e0478aab admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:11.750104 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:11.875357 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 970/1950] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:11:11 2022] POST /identity/v3/users => generated 300 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 22:11:11.876712 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155e4f18-2be3-40be-af25-0fbf387baf54 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:11.907277 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155e4f18-2be3-40be-af25-0fbf387baf54 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:11.907369 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e6e283-7e7e-4bdc-bf85-2159e0478aab admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:11.909194 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-47e6e283-7e7e-4bdc-bf85-2159e0478aab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:11.920645 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db0b5a9-f5b6-48ba-b9d6-29eda6378cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:11.921362 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db0b5a9-f5b6-48ba-b9d6-29eda6378cb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:11.921709 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db0b5a9-f5b6-48ba-b9d6-29eda6378cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:11.922050 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db0b5a9-f5b6-48ba-b9d6-29eda6378cb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:11.922792 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db0b5a9-f5b6-48ba-b9d6-29eda6378cb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:11.924055 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e6e283-7e7e-4bdc-bf85-2159e0478aab admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:11.924578 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db0b5a9-f5b6-48ba-b9d6-29eda6378cb1 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:11.929093 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6fb3cb-779f-4e4d-9c54-5424c1eb8eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:11.929926 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6fb3cb-779f-4e4d-9c54-5424c1eb8eaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:11.930218 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6fb3cb-779f-4e4d-9c54-5424c1eb8eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:11.930495 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6fb3cb-779f-4e4d-9c54-5424c1eb8eaf admin admin] PATH_INFO: `/v3/projects/e95bcdc3503c4137ac0c949ab5964a15` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:11.935404 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6fb3cb-779f-4e4d-9c54-5424c1eb8eaf admin admin] RBAC: Authorizing `identity:delete_project(project_id=e95bcdc3503c4137ac0c949ab5964a15)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:11.937444 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6fb3cb-779f-4e4d-9c54-5424c1eb8eaf admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:11.944273 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 981/1951] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:11:11 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:11:11.945546 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db0b5a9-f5b6-48ba-b9d6-29eda6378cb1 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:11.956212 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db0b5a9-f5b6-48ba-b9d6-29eda6378cb1 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:11.961618 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f2eb93-a0ee-4901-8ba8-9b7142f19fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:11.962472 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f2eb93-a0ee-4901-8ba8-9b7142f19fbd admin admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:11.962751 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f2eb93-a0ee-4901-8ba8-9b7142f19fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:11.963023 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f2eb93-a0ee-4901-8ba8-9b7142f19fbd admin admin] PATH_INFO: `/v3/projects/568cb563d8674e89bc3bff229634a6cb/users/483606940ff44277b3a0f1a78c4ef60d/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:11.979744 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f2eb93-a0ee-4901-8ba8-9b7142f19fbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=568cb563d8674e89bc3bff229634a6cb, user_id=483606940ff44277b3a0f1a78c4ef60d, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:11.982526 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f2eb93-a0ee-4901-8ba8-9b7142f19fbd admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:12.001384 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:12.070215 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 982/1952] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:11:11 2022] PUT /identity/v3/projects/568cb563d8674e89bc3bff229634a6cb/users/483606940ff44277b3a0f1a78c4ef60d/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:11:12.074786 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f2eb93-a0ee-4901-8ba8-9b7142f19fbd admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:12.078779 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-0a6fb3cb-779f-4e4d-9c54-5424c1eb8eaf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e95bcdc3503c4137ac0c949ab5964a15', 'request_id': 'req-0a6fb3cb-779f-4e4d-9c54-5424c1eb8eaf'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:12.138407 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f2eb93-a0ee-4901-8ba8-9b7142f19fbd admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:12.148435 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-0a6fb3cb-779f-4e4d-9c54-5424c1eb8eaf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e95bcdc3503c4137ac0c949ab5964a15', 'request_id': 'req-0a6fb3cb-779f-4e4d-9c54-5424c1eb8eaf'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:12.154699 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-0a6fb3cb-779f-4e4d-9c54-5424c1eb8eaf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e95bcdc3503c4137ac0c949ab5964a15', 'request_id': 'req-0a6fb3cb-779f-4e4d-9c54-5424c1eb8eaf'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:12.167032 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-0a6fb3cb-779f-4e4d-9c54-5424c1eb8eaf admin admin] The token cache is being invalidate because project e95bcdc3503c4137ac0c949ab5964a15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:11:12.167439 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-0a6fb3cb-779f-4e4d-9c54-5424c1eb8eaf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:12.168118 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-0a6fb3cb-779f-4e4d-9c54-5424c1eb8eaf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:12.174229 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 971/1953] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:11:11 2022] DELETE /identity/v3/projects/e95bcdc3503c4137ac0c949ab5964a15 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 22:11:12.175035 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9ba6eb-c527-45fb-987a-8a4ef52de440 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:12.175317 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9ba6eb-c527-45fb-987a-8a4ef52de440 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:12.175591 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9ba6eb-c527-45fb-987a-8a4ef52de440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:12.195262 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78c2049-1239-4245-b516-845c78214aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:12.195481 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-be9ba6eb-c527-45fb-987a-8a4ef52de440 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:11:12.195880 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78c2049-1239-4245-b516-845c78214aa4 admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:12.196163 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78c2049-1239-4245-b516-845c78214aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:12.196475 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78c2049-1239-4245-b516-845c78214aa4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:12.196614 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-be9ba6eb-c527-45fb-987a-8a4ef52de440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:12.197124 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78c2049-1239-4245-b516-845c78214aa4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:12.198714 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78c2049-1239-4245-b516-845c78214aa4 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:12.205436 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 972/1954] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:11:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 22:11:12.209736 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 983/1955] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:11:12 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:11:12.213564 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78c2049-1239-4245-b516-845c78214aa4 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:12.213716 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcea5efc-b352-4862-a246-22efaa82e2ac None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:12.213957 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcea5efc-b352-4862-a246-22efaa82e2ac None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:12.214281 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcea5efc-b352-4862-a246-22efaa82e2ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:12.215282 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-e78c2049-1239-4245-b516-845c78214aa4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:12.228236 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78c2049-1239-4245-b516-845c78214aa4 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:12.233170 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9e11df-28e1-40a1-92b3-cec7494cf236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:12.233833 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9e11df-28e1-40a1-92b3-cec7494cf236 admin admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:12.237880 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9e11df-28e1-40a1-92b3-cec7494cf236 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:12.237880 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9e11df-28e1-40a1-92b3-cec7494cf236 admin admin] PATH_INFO: `/v3/projects/568cb563d8674e89bc3bff229634a6cb/users/483606940ff44277b3a0f1a78c4ef60d/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:12.238065 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-bcea5efc-b352-4862-a246-22efaa82e2ac None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:11:12.238065 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-bcea5efc-b352-4862-a246-22efaa82e2ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:12.238223 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9e11df-28e1-40a1-92b3-cec7494cf236 admin admin] RBAC: Authorizing `identity:create_grant(project_id=568cb563d8674e89bc3bff229634a6cb, user_id=483606940ff44277b3a0f1a78c4ef60d, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:12.238931 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9e11df-28e1-40a1-92b3-cec7494cf236 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:12.244680 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 973/1956] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:11:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:11:12.249517 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcea5efc-b352-4862-a246-22efaa82e2ac None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:12.250358 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 984/1957] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:11:12 2022] PUT /identity/v3/projects/568cb563d8674e89bc3bff229634a6cb/users/483606940ff44277b3a0f1a78c4ef60d/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 22:11:12.259984 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcea5efc-b352-4862-a246-22efaa82e2ac None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:12.261293 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d2354e-93c6-4ebf-af7b-b785676e9901 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:12.261687 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d2354e-93c6-4ebf-af7b-b785676e9901 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:12.262012 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d2354e-93c6-4ebf-af7b-b785676e9901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:12.290366 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-99d2354e-93c6-4ebf-af7b-b785676e9901 None None] MFA Rules not processed for user `483606940ff44277b3a0f1a78c4ef60d`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:11:12.291586 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-99d2354e-93c6-4ebf-af7b-b785676e9901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:12.316555 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997cd27a-18fc-4507-901e-a7098520cd40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:12.317157 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997cd27a-18fc-4507-901e-a7098520cd40 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:12.317433 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997cd27a-18fc-4507-901e-a7098520cd40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:12.317711 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997cd27a-18fc-4507-901e-a7098520cd40 admin admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:12.318348 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997cd27a-18fc-4507-901e-a7098520cd40 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 22:11:12.318648 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997cd27a-18fc-4507-901e-a7098520cd40 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:12.320350 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997cd27a-18fc-4507-901e-a7098520cd40 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:12.335190 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 974/1958] 10.222.0.18 () {58 vars in 1153 bytes} [Sat Aug 6 22:11:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 22:11:12.336601 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997cd27a-18fc-4507-901e-a7098520cd40 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:12.347111 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997cd27a-18fc-4507-901e-a7098520cd40 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:12.349552 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:12.352339 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70ade42-268c-4912-89a8-20de884d4233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:12.352996 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70ade42-268c-4912-89a8-20de884d4233 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:12.353274 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70ade42-268c-4912-89a8-20de884d4233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:12.353546 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70ade42-268c-4912-89a8-20de884d4233 admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:12.354169 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70ade42-268c-4912-89a8-20de884d4233 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:12.356070 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70ade42-268c-4912-89a8-20de884d4233 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:12.369467 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 985/1959] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:11:12 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 22:11:12.380896 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d2354e-93c6-4ebf-af7b-b785676e9901 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:12.382398 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-99d2354e-93c6-4ebf-af7b-b785676e9901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:12.444646 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d2354e-93c6-4ebf-af7b-b785676e9901 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:12.448571 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7418bb-fe11-45d8-93db-f226084d5797 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa71c71de263478da234f3c5716a6e5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:12.449192 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7418bb-fe11-45d8-93db-f226084d5797 service cinder] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:12.449470 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7418bb-fe11-45d8-93db-f226084d5797 service cinder] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:12.449743 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7418bb-fe11-45d8-93db-f226084d5797 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:12.451033 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 975/1960] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:11:12 2022] POST /identity/v3/projects => generated 377 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:11:12.455200 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70ade42-268c-4912-89a8-20de884d4233 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:12.463781 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7418bb-fe11-45d8-93db-f226084d5797 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:12.465375 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7418bb-fe11-45d8-93db-f226084d5797 service cinder] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:12.477279 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 986/1961] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:11:12 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 22:11:12.532743 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70ade42-268c-4912-89a8-20de884d4233 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:12.541011 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef07949-7036-4134-a63d-8ecad18d0be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:12.541011 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef07949-7036-4134-a63d-8ecad18d0be0 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:12.541011 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef07949-7036-4134-a63d-8ecad18d0be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:12.541011 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef07949-7036-4134-a63d-8ecad18d0be0 admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:12.541543 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef07949-7036-4134-a63d-8ecad18d0be0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:12.546419 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef07949-7036-4134-a63d-8ecad18d0be0 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:12.572887 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7418bb-fe11-45d8-93db-f226084d5797 service cinder] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:12.594464 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7418bb-fe11-45d8-93db-f226084d5797 service cinder] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:12.600050 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e56fb8-dc3e-492b-9b2b-d2c02cbb4a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:12.600728 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e56fb8-dc3e-492b-9b2b-d2c02cbb4a37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:12.601277 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e56fb8-dc3e-492b-9b2b-d2c02cbb4a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:12.610858 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e56fb8-dc3e-492b-9b2b-d2c02cbb4a37 admin admin] PATH_INFO: `/v3/users/97c467309b774af99980909e5a94841f` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:12.610858 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e56fb8-dc3e-492b-9b2b-d2c02cbb4a37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97c467309b774af99980909e5a94841f)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:12.610858 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e56fb8-dc3e-492b-9b2b-d2c02cbb4a37 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:12.664266 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 976/1962] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:11:12 2022] POST /identity/v3/users => generated 289 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:11:12.668721 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef07949-7036-4134-a63d-8ecad18d0be0 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:12.678466 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef07949-7036-4134-a63d-8ecad18d0be0 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:12.683273 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a4e818-dac2-4a3c-a235-fd876b5c7988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:12.683872 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a4e818-dac2-4a3c-a235-fd876b5c7988 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:12.684150 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a4e818-dac2-4a3c-a235-fd876b5c7988 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:12.684641 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a4e818-dac2-4a3c-a235-fd876b5c7988 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:12.685467 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a4e818-dac2-4a3c-a235-fd876b5c7988 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:12.687066 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a4e818-dac2-4a3c-a235-fd876b5c7988 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:12.699321 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 977/1963] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:11:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:11:12.707746 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a4e818-dac2-4a3c-a235-fd876b5c7988 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:12.718764 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a4e818-dac2-4a3c-a235-fd876b5c7988 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:12.723289 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81735166-1919-4609-b6ee-a9e626407ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:12.723870 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81735166-1919-4609-b6ee-a9e626407ec6 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:12.724153 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81735166-1919-4609-b6ee-a9e626407ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:12.724452 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81735166-1919-4609-b6ee-a9e626407ec6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:12.725085 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81735166-1919-4609-b6ee-a9e626407ec6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:12.727009 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81735166-1919-4609-b6ee-a9e626407ec6 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:12.737604 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 978/1964] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:11:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 22:11:12.743118 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81735166-1919-4609-b6ee-a9e626407ec6 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:12.753285 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81735166-1919-4609-b6ee-a9e626407ec6 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:12.757762 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eebc04b-819f-4b0f-9650-7cb68e8e5c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:12.758378 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eebc04b-819f-4b0f-9650-7cb68e8e5c94 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:12.758658 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eebc04b-819f-4b0f-9650-7cb68e8e5c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:12.758934 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eebc04b-819f-4b0f-9650-7cb68e8e5c94 admin admin] PATH_INFO: `/v3/projects/e49c8f3f6e10431282d64c4036246499/users/1db1473cf322488dbb7b9f26419207fa/roles/56e4a96094e9434cbbafb22579c034f5` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:12.774894 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eebc04b-819f-4b0f-9650-7cb68e8e5c94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e49c8f3f6e10431282d64c4036246499, user_id=1db1473cf322488dbb7b9f26419207fa, role_id=56e4a96094e9434cbbafb22579c034f5)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:12.777641 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eebc04b-819f-4b0f-9650-7cb68e8e5c94 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:12.900449 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-94e56fb8-dc3e-492b-9b2b-d2c02cbb4a37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97c467309b774af99980909e5a94841f'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:12.912427 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-94e56fb8-dc3e-492b-9b2b-d2c02cbb4a37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97c467309b774af99980909e5a94841f'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:12.921503 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 979/1965] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:11:12 2022] PUT /identity/v3/projects/e49c8f3f6e10431282d64c4036246499/users/1db1473cf322488dbb7b9f26419207fa/roles/56e4a96094e9434cbbafb22579c034f5 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 22:11:12.931526 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e0d46-3684-44e1-a8dc-f95cc0132fe8 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:12.931784 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e0d46-3684-44e1-a8dc-f95cc0132fe8 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:12.932054 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e0d46-3684-44e1-a8dc-f95cc0132fe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:12.959835 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-749e0d46-3684-44e1-a8dc-f95cc0132fe8 None None] MFA Rules not processed for user `1db1473cf322488dbb7b9f26419207fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:11:12.961051 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-749e0d46-3684-44e1-a8dc-f95cc0132fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:13.006961 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:13.025891 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 980/1966] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:11:12 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 22:11:13.030709 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749e0d46-3684-44e1-a8dc-f95cc0132fe8 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:13.046548 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-94e56fb8-dc3e-492b-9b2b-d2c02cbb4a37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97c467309b774af99980909e5a94841f'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:13.055253 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-94e56fb8-dc3e-492b-9b2b-d2c02cbb4a37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97c467309b774af99980909e5a94841f'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:13.055929 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-94e56fb8-dc3e-492b-9b2b-d2c02cbb4a37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97c467309b774af99980909e5a94841f'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:13.062708 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 987/1967] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:11:12 2022] DELETE /identity/v3/users/97c467309b774af99980909e5a94841f => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 22:11:13.067185 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdfea69-7722-4f7c-8295-d76c47cb8617 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:13.067623 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdfea69-7722-4f7c-8295-d76c47cb8617 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:13.068094 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdfea69-7722-4f7c-8295-d76c47cb8617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:13.086994 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:13.088622 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-3fdfea69-7722-4f7c-8295-d76c47cb8617 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:11:13.089921 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-3fdfea69-7722-4f7c-8295-d76c47cb8617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:13.096066 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749e0d46-3684-44e1-a8dc-f95cc0132fe8 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:13.152115 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 988/1968] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:11:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 22:11:13.157155 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4aef71-e976-4738-b533-1989693ebe82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:13.157770 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4aef71-e976-4738-b533-1989693ebe82 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:13.158046 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4aef71-e976-4738-b533-1989693ebe82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:13.158322 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4aef71-e976-4738-b533-1989693ebe82 admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:13.158950 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4aef71-e976-4738-b533-1989693ebe82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:13.160887 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4aef71-e976-4738-b533-1989693ebe82 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:13.164558 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdfea69-7722-4f7c-8295-d76c47cb8617 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:13.165956 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-3fdfea69-7722-4f7c-8295-d76c47cb8617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:13.226379 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdfea69-7722-4f7c-8295-d76c47cb8617 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:13.230084 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c178e90c-4857-44b1-a4a0-f92f35547a8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:13.230692 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c178e90c-4857-44b1-a4a0-f92f35547a8c service neutron] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:13.230970 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c178e90c-4857-44b1-a4a0-f92f35547a8c service neutron] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:13.231775 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c178e90c-4857-44b1-a4a0-f92f35547a8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:13.242034 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c178e90c-4857-44b1-a4a0-f92f35547a8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:13.243446 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c178e90c-4857-44b1-a4a0-f92f35547a8c service neutron] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:13.253731 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 989/1969] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:11:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:11:13.294296 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 981/1970] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:11:13 2022] POST /identity/v3/projects => generated 377 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 22:11:13.302655 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c178e90c-4857-44b1-a4a0-f92f35547a8c service neutron] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:13.302655 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-c178e90c-4857-44b1-a4a0-f92f35547a8c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:13.313267 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c178e90c-4857-44b1-a4a0-f92f35547a8c service neutron] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:13.333077 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:13.371866 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b920181-4e50-4497-a64f-8177a8bfb63c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:13.373353 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b920181-4e50-4497-a64f-8177a8bfb63c admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:13.373686 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b920181-4e50-4497-a64f-8177a8bfb63c admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:13.374028 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b920181-4e50-4497-a64f-8177a8bfb63c admin admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:13.374747 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b920181-4e50-4497-a64f-8177a8bfb63c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:13.376814 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b920181-4e50-4497-a64f-8177a8bfb63c admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:13.461755 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 990/1971] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:11:13 2022] POST /identity/v3/users => generated 289 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:11:13.462921 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4aef71-e976-4738-b533-1989693ebe82 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:13.477115 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4aef71-e976-4738-b533-1989693ebe82 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:13.482963 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:13.483969 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854d2e5b-86b1-460a-95ed-bbc3be8ff3f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:13.484627 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854d2e5b-86b1-460a-95ed-bbc3be8ff3f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:13.484909 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854d2e5b-86b1-460a-95ed-bbc3be8ff3f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:13.485238 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854d2e5b-86b1-460a-95ed-bbc3be8ff3f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:13.485894 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854d2e5b-86b1-460a-95ed-bbc3be8ff3f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:13.487766 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854d2e5b-86b1-460a-95ed-bbc3be8ff3f1 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:13.502709 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 982/1972] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:11:13 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:11:13.503931 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b920181-4e50-4497-a64f-8177a8bfb63c admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:13.514312 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b920181-4e50-4497-a64f-8177a8bfb63c admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:13.519382 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d7897d-d38d-43fb-a995-e4a0091eca68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:13.520199 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d7897d-d38d-43fb-a995-e4a0091eca68 admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:13.520609 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d7897d-d38d-43fb-a995-e4a0091eca68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:13.520960 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d7897d-d38d-43fb-a995-e4a0091eca68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:13.521688 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d7897d-d38d-43fb-a995-e4a0091eca68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:13.523398 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d7897d-d38d-43fb-a995-e4a0091eca68 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:13.535113 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 991/1973] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:11:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:11:13.539564 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854d2e5b-86b1-460a-95ed-bbc3be8ff3f1 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:13.551623 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854d2e5b-86b1-460a-95ed-bbc3be8ff3f1 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:13.558389 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8912386a-7f04-45ef-98ef-67cfb3697400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:13.559472 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8912386a-7f04-45ef-98ef-67cfb3697400 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:13.559833 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8912386a-7f04-45ef-98ef-67cfb3697400 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:13.560553 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8912386a-7f04-45ef-98ef-67cfb3697400 admin admin] PATH_INFO: `/v3/projects/bbff8826d2d74874a46fdef774001fa8/users/493c82a5168a44a595d2bb5cac94dbbb/roles/56e4a96094e9434cbbafb22579c034f5` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:13.581482 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8912386a-7f04-45ef-98ef-67cfb3697400 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbff8826d2d74874a46fdef774001fa8, user_id=493c82a5168a44a595d2bb5cac94dbbb, role_id=56e4a96094e9434cbbafb22579c034f5)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:13.584160 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8912386a-7f04-45ef-98ef-67cfb3697400 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:13.726216 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 983/1974] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:11:13 2022] PUT /identity/v3/projects/bbff8826d2d74874a46fdef774001fa8/users/493c82a5168a44a595d2bb5cac94dbbb/roles/56e4a96094e9434cbbafb22579c034f5 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:11:13.744369 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f85ff-5d4c-4a7a-8bc5-ea2bc11ca81b None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:13.744920 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f85ff-5d4c-4a7a-8bc5-ea2bc11ca81b None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:13.745334 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f85ff-5d4c-4a7a-8bc5-ea2bc11ca81b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:13.746905 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:13.756605 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8912386a-7f04-45ef-98ef-67cfb3697400 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:13.758533 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-8912386a-7f04-45ef-98ef-67cfb3697400 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:13.801080 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-dc7f85ff-5d4c-4a7a-8bc5-ea2bc11ca81b None None] MFA Rules not processed for user `493c82a5168a44a595d2bb5cac94dbbb`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:11:13.802599 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-dc7f85ff-5d4c-4a7a-8bc5-ea2bc11ca81b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:13.834309 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8912386a-7f04-45ef-98ef-67cfb3697400 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:13.839310 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3520e194-8176-4bf0-acdc-568764dcc7ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:13.840144 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3520e194-8176-4bf0-acdc-568764dcc7ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:13.840811 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3520e194-8176-4bf0-acdc-568764dcc7ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:13.841416 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3520e194-8176-4bf0-acdc-568764dcc7ff admin admin] PATH_INFO: `/v3/projects/b54123196a8f46ae9298e89e4262c2d2` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:13.842687 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3520e194-8176-4bf0-acdc-568764dcc7ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=b54123196a8f46ae9298e89e4262c2d2)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:13.845004 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3520e194-8176-4bf0-acdc-568764dcc7ff admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:13.865173 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:13.911496 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 992/1975] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:11:13 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:11:13.915949 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7f85ff-5d4c-4a7a-8bc5-ea2bc11ca81b None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:13.929388 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7f85ff-5d4c-4a7a-8bc5-ea2bc11ca81b None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:13.936317 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a33055-1320-42e8-9b8f-76ae374ea999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:13.937053 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a33055-1320-42e8-9b8f-76ae374ea999 admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:13.937332 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a33055-1320-42e8-9b8f-76ae374ea999 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:13.937693 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a33055-1320-42e8-9b8f-76ae374ea999 admin admin] PATH_INFO: `/v3/projects` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:13.938260 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a33055-1320-42e8-9b8f-76ae374ea999 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:13.940193 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a33055-1320-42e8-9b8f-76ae374ea999 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:14.001191 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-3520e194-8176-4bf0-acdc-568764dcc7ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b54123196a8f46ae9298e89e4262c2d2', 'request_id': 'req-3520e194-8176-4bf0-acdc-568764dcc7ff'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:14.042031 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 993/1976] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:11:13 2022] POST /identity/v3/projects => generated 377 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 22:11:14.046228 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a33055-1320-42e8-9b8f-76ae374ea999 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:14.079654 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-3520e194-8176-4bf0-acdc-568764dcc7ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b54123196a8f46ae9298e89e4262c2d2', 'request_id': 'req-3520e194-8176-4bf0-acdc-568764dcc7ff'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:14.085836 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-3520e194-8176-4bf0-acdc-568764dcc7ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b54123196a8f46ae9298e89e4262c2d2', 'request_id': 'req-3520e194-8176-4bf0-acdc-568764dcc7ff'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:14.098012 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-3520e194-8176-4bf0-acdc-568764dcc7ff admin admin] The token cache is being invalidate because project b54123196a8f46ae9298e89e4262c2d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:11:14.098307 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-3520e194-8176-4bf0-acdc-568764dcc7ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:14.098977 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-3520e194-8176-4bf0-acdc-568764dcc7ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:14.100744 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 984/1977] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:11:13 2022] DELETE /identity/v3/projects/b54123196a8f46ae9298e89e4262c2d2 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 22:11:14.104722 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3520e194-8176-4bf0-acdc-568764dcc7ff admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:14.106121 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-3520e194-8176-4bf0-acdc-568764dcc7ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:14.107669 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a33055-1320-42e8-9b8f-76ae374ea999 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:14.112508 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e522a80d-5fc0-461b-8578-22fc7d667c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:14.113247 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e522a80d-5fc0-461b-8578-22fc7d667c58 admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:14.113610 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e522a80d-5fc0-461b-8578-22fc7d667c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:14.113888 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e522a80d-5fc0-461b-8578-22fc7d667c58 admin admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:14.114662 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e522a80d-5fc0-461b-8578-22fc7d667c58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:14.116601 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e522a80d-5fc0-461b-8578-22fc7d667c58 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:14.167992 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3520e194-8176-4bf0-acdc-568764dcc7ff admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:14.172744 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99e7ddb-4187-41b8-b875-0baf432f9a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:14.173363 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99e7ddb-4187-41b8-b875-0baf432f9a4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:14.173637 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99e7ddb-4187-41b8-b875-0baf432f9a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:14.173905 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99e7ddb-4187-41b8-b875-0baf432f9a4e admin admin] PATH_INFO: `/v3/users/483606940ff44277b3a0f1a78c4ef60d` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:14.175110 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99e7ddb-4187-41b8-b875-0baf432f9a4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=483606940ff44277b3a0f1a78c4ef60d)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:14.177072 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99e7ddb-4187-41b8-b875-0baf432f9a4e admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:14.218691 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 994/1978] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:11:14 2022] POST /identity/v3/users => generated 295 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 22:11:14.222768 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e522a80d-5fc0-461b-8578-22fc7d667c58 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:14.224343 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-e522a80d-5fc0-461b-8578-22fc7d667c58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:14.236325 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e522a80d-5fc0-461b-8578-22fc7d667c58 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:14.240967 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9326fd3a-ee14-4296-a12c-4d85df95363d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:14.241550 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9326fd3a-ee14-4296-a12c-4d85df95363d admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:14.241846 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9326fd3a-ee14-4296-a12c-4d85df95363d admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:14.242106 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9326fd3a-ee14-4296-a12c-4d85df95363d admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:14.242738 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9326fd3a-ee14-4296-a12c-4d85df95363d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:14.244293 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9326fd3a-ee14-4296-a12c-4d85df95363d admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:14.255960 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 995/1979] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:11:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 22:11:14.259824 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9326fd3a-ee14-4296-a12c-4d85df95363d admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:14.270290 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9326fd3a-ee14-4296-a12c-4d85df95363d admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:14.275593 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cdb43b-5ecf-4057-946e-25b25c8de0bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:14.276134 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cdb43b-5ecf-4057-946e-25b25c8de0bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:14.276445 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cdb43b-5ecf-4057-946e-25b25c8de0bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:14.276711 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cdb43b-5ecf-4057-946e-25b25c8de0bf admin admin] PATH_INFO: `/v3/projects/9f3e1902761e4ffd962c750c74740891/users/2c3a6bf7a11a41c19a027738b6fe32e9/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:14.279196 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:14.292127 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cdb43b-5ecf-4057-946e-25b25c8de0bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f3e1902761e4ffd962c750c74740891, user_id=2c3a6bf7a11a41c19a027738b6fe32e9, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:14.299898 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cdb43b-5ecf-4057-946e-25b25c8de0bf admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:14.372780 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 996/1980] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:11:14 2022] PUT /identity/v3/projects/9f3e1902761e4ffd962c750c74740891/users/2c3a6bf7a11a41c19a027738b6fe32e9/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:11:14.377280 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cdb43b-5ecf-4057-946e-25b25c8de0bf admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:14.382650 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-f99e7ddb-4187-41b8-b875-0baf432f9a4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '483606940ff44277b3a0f1a78c4ef60d'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:14.397081 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-f99e7ddb-4187-41b8-b875-0baf432f9a4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '483606940ff44277b3a0f1a78c4ef60d'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:14.405144 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-f99e7ddb-4187-41b8-b875-0baf432f9a4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '483606940ff44277b3a0f1a78c4ef60d'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:14.405868 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-f99e7ddb-4187-41b8-b875-0baf432f9a4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '483606940ff44277b3a0f1a78c4ef60d'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:14.406466 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-f99e7ddb-4187-41b8-b875-0baf432f9a4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '483606940ff44277b3a0f1a78c4ef60d'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:14.451179 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cdb43b-5ecf-4057-946e-25b25c8de0bf admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:14.455851 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f657352b-53bd-4ebc-aa06-d6a740c7cf70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:14.456471 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f657352b-53bd-4ebc-aa06-d6a740c7cf70 admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:14.456750 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f657352b-53bd-4ebc-aa06-d6a740c7cf70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:14.457018 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f657352b-53bd-4ebc-aa06-d6a740c7cf70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:14.457652 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f657352b-53bd-4ebc-aa06-d6a740c7cf70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:14.459230 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f657352b-53bd-4ebc-aa06-d6a740c7cf70 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:14.469446 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 997/1981] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:11:14 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:11:14.473432 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f657352b-53bd-4ebc-aa06-d6a740c7cf70 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:14.475074 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-f657352b-53bd-4ebc-aa06-d6a740c7cf70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:14.489062 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f657352b-53bd-4ebc-aa06-d6a740c7cf70 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:14.494160 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3125c7c5-0495-4289-89a7-783c81e8401c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:14.494806 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3125c7c5-0495-4289-89a7-783c81e8401c admin admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:14.495091 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3125c7c5-0495-4289-89a7-783c81e8401c admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:14.495362 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3125c7c5-0495-4289-89a7-783c81e8401c admin admin] PATH_INFO: `/v3/projects/9f3e1902761e4ffd962c750c74740891/users/2c3a6bf7a11a41c19a027738b6fe32e9/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:14.497527 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3125c7c5-0495-4289-89a7-783c81e8401c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f3e1902761e4ffd962c750c74740891, user_id=2c3a6bf7a11a41c19a027738b6fe32e9, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:14.500141 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3125c7c5-0495-4289-89a7-783c81e8401c admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:14.511224 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 998/1982] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:11:14 2022] PUT /identity/v3/projects/9f3e1902761e4ffd962c750c74740891/users/2c3a6bf7a11a41c19a027738b6fe32e9/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 22:11:14.527615 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e984e02-35ef-4bbc-983c-4a0a3328918e None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:14.528392 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e984e02-35ef-4bbc-983c-4a0a3328918e None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:14.528701 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e984e02-35ef-4bbc-983c-4a0a3328918e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:14.557619 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 985/1983] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:11:14 2022] DELETE /identity/v3/users/483606940ff44277b3a0f1a78c4ef60d => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 22:11:14.557795 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-6e984e02-35ef-4bbc-983c-4a0a3328918e None None] MFA Rules not processed for user `2c3a6bf7a11a41c19a027738b6fe32e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:11:14.558759 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-6e984e02-35ef-4bbc-983c-4a0a3328918e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:14.584854 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=88,keystone:DELETE=8,keystone:INSERT=4 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:14.636177 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 999/1984] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:11:14 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 22:11:14.648027 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99e7ddb-4187-41b8-b875-0baf432f9a4e admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:14.649542 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-f99e7ddb-4187-41b8-b875-0baf432f9a4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:14.706710 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99e7ddb-4187-41b8-b875-0baf432f9a4e admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:14.710358 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b48f57-5a40-482d-9bde-f3fb55c2c857 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa71c71de263478da234f3c5716a6e5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:14.710956 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b48f57-5a40-482d-9bde-f3fb55c2c857 service cinder] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:14.711235 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b48f57-5a40-482d-9bde-f3fb55c2c857 service cinder] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:14.711504 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b48f57-5a40-482d-9bde-f3fb55c2c857 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:14.721826 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b48f57-5a40-482d-9bde-f3fb55c2c857 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:14.723325 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b48f57-5a40-482d-9bde-f3fb55c2c857 service cinder] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:14.733149 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 986/1985] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:11:14 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:11:14.867116 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e984e02-35ef-4bbc-983c-4a0a3328918e None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:14.868770 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-6e984e02-35ef-4bbc-983c-4a0a3328918e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:14.876355 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:14.882175 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e984e02-35ef-4bbc-983c-4a0a3328918e None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:14.886160 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8eae2b-3741-45b1-8dd1-6802e48c7a8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa71c71de263478da234f3c5716a6e5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:14.886769 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8eae2b-3741-45b1-8dd1-6802e48c7a8d service cinder] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:14.887044 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8eae2b-3741-45b1-8dd1-6802e48c7a8d service cinder] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:14.887430 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8eae2b-3741-45b1-8dd1-6802e48c7a8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:14.889196 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-0b8eae2b-3741-45b1-8dd1-6802e48c7a8d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:14.955042 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8eae2b-3741-45b1-8dd1-6802e48c7a8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:14.956569 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8eae2b-3741-45b1-8dd1-6802e48c7a8d service cinder] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:14.967049 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1000/1986] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:11:14 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:11:14.994459 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b48f57-5a40-482d-9bde-f3fb55c2c857 service cinder] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:14.997739 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-52b48f57-5a40-482d-9bde-f3fb55c2c857 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:15.075740 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:15.109740 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b48f57-5a40-482d-9bde-f3fb55c2c857 service cinder] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:15.114717 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8654cf57-814e-44d4-bce3-2153dd4852d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:15.115659 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8654cf57-814e-44d4-bce3-2153dd4852d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:15.116274 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8654cf57-814e-44d4-bce3-2153dd4852d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:15.116588 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8654cf57-814e-44d4-bce3-2153dd4852d9 admin admin] PATH_INFO: `/v3/projects/568cb563d8674e89bc3bff229634a6cb` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:15.117885 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8654cf57-814e-44d4-bce3-2153dd4852d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=568cb563d8674e89bc3bff229634a6cb)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:15.120366 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8654cf57-814e-44d4-bce3-2153dd4852d9 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:15.155514 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:15.229485 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-8654cf57-814e-44d4-bce3-2153dd4852d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '568cb563d8674e89bc3bff229634a6cb', 'request_id': 'req-8654cf57-814e-44d4-bce3-2153dd4852d9'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:15.337663 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-8654cf57-814e-44d4-bce3-2153dd4852d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '568cb563d8674e89bc3bff229634a6cb', 'request_id': 'req-8654cf57-814e-44d4-bce3-2153dd4852d9'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:15.348744 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-8654cf57-814e-44d4-bce3-2153dd4852d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '568cb563d8674e89bc3bff229634a6cb', 'request_id': 'req-8654cf57-814e-44d4-bce3-2153dd4852d9'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:15.355359 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-8654cf57-814e-44d4-bce3-2153dd4852d9 admin admin] The token cache is being invalidate because project 568cb563d8674e89bc3bff229634a6cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:11:15.355648 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-8654cf57-814e-44d4-bce3-2153dd4852d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:15.356934 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-8654cf57-814e-44d4-bce3-2153dd4852d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:15.358771 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 987/1987] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:11:14 2022] DELETE /identity/v3/projects/568cb563d8674e89bc3bff229634a6cb => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:11:15.363574 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62fb701-f12d-4ea7-a160-57edb5a91f21 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:15.363857 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62fb701-f12d-4ea7-a160-57edb5a91f21 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:15.364127 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62fb701-f12d-4ea7-a160-57edb5a91f21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:15.384899 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-c62fb701-f12d-4ea7-a160-57edb5a91f21 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:11:15.388857 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-c62fb701-f12d-4ea7-a160-57edb5a91f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:15.446633 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1001/1988] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:11:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:11:15.454505 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea9e073-7ce0-4156-ae0b-d86604916f6e None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:15.454905 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea9e073-7ce0-4156-ae0b-d86604916f6e None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:15.455184 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea9e073-7ce0-4156-ae0b-d86604916f6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:15.475423 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-7ea9e073-7ce0-4156-ae0b-d86604916f6e None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:11:15.476997 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-7ea9e073-7ce0-4156-ae0b-d86604916f6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:15.487161 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 988/1989] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:11:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 22:11:15.492849 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62fb701-f12d-4ea7-a160-57edb5a91f21 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:15.504869 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62fb701-f12d-4ea7-a160-57edb5a91f21 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:15.510988 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fa5c79-eb24-49d1-b078-5dd77b75b191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:15.511776 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fa5c79-eb24-49d1-b078-5dd77b75b191 admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:15.512220 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fa5c79-eb24-49d1-b078-5dd77b75b191 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:15.512542 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fa5c79-eb24-49d1-b078-5dd77b75b191 admin admin] PATH_INFO: `/v3/domains` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:15.513201 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fa5c79-eb24-49d1-b078-5dd77b75b191 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 22:11:15.513500 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fa5c79-eb24-49d1-b078-5dd77b75b191 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:15.515088 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fa5c79-eb24-49d1-b078-5dd77b75b191 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:15.528063 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1002/1990] 10.222.0.18 () {58 vars in 1153 bytes} [Sat Aug 6 22:11:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 22:11:15.532449 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea9e073-7ce0-4156-ae0b-d86604916f6e None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:15.542491 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea9e073-7ce0-4156-ae0b-d86604916f6e None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:15.554758 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac581d8-5a63-4a15-866a-6ae4187beab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:15.556839 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac581d8-5a63-4a15-866a-6ae4187beab0 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:15.557219 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac581d8-5a63-4a15-866a-6ae4187beab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:15.557615 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac581d8-5a63-4a15-866a-6ae4187beab0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:15.558522 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac581d8-5a63-4a15-866a-6ae4187beab0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:15.560954 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac581d8-5a63-4a15-866a-6ae4187beab0 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:15.630196 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:15.665192 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 989/1991] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:11:15 2022] POST /identity/v3/projects => generated 387 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:11:15.670030 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fa5c79-eb24-49d1-b078-5dd77b75b191 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:15.680853 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fa5c79-eb24-49d1-b078-5dd77b75b191 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:15.727584 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:15.743052 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2d021b-6898-45b3-82ea-6d81bc95eaab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:15.743660 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d021b-6898-45b3-82ea-6d81bc95eaab admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:15.743941 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d021b-6898-45b3-82ea-6d81bc95eaab admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:15.744215 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d021b-6898-45b3-82ea-6d81bc95eaab admin admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:15.744893 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2d021b-6898-45b3-82ea-6d81bc95eaab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:15.746937 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2d021b-6898-45b3-82ea-6d81bc95eaab admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:15.869478 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1003/1992] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:11:15 2022] POST /identity/v3/users => generated 294 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:11:15.873617 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac581d8-5a63-4a15-866a-6ae4187beab0 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:15.884484 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac581d8-5a63-4a15-866a-6ae4187beab0 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:15.889751 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cf48f3-aa94-43b1-9a1e-25a049d52213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:15.890562 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cf48f3-aa94-43b1-9a1e-25a049d52213 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:15.890844 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cf48f3-aa94-43b1-9a1e-25a049d52213 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:15.891111 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cf48f3-aa94-43b1-9a1e-25a049d52213 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:15.891748 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cf48f3-aa94-43b1-9a1e-25a049d52213 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:15.893551 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cf48f3-aa94-43b1-9a1e-25a049d52213 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:15.905519 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 990/1993] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:11:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:11:15.913372 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2d021b-6898-45b3-82ea-6d81bc95eaab admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:15.924237 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2d021b-6898-45b3-82ea-6d81bc95eaab admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:15.929105 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f78cb38-e283-4f15-8b06-eaae4e833d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:15.929954 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f78cb38-e283-4f15-8b06-eaae4e833d36 admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:15.930084 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f78cb38-e283-4f15-8b06-eaae4e833d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:15.930366 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f78cb38-e283-4f15-8b06-eaae4e833d36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:15.932385 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f78cb38-e283-4f15-8b06-eaae4e833d36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:15.932718 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f78cb38-e283-4f15-8b06-eaae4e833d36 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:15.943175 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1004/1994] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:11:15 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:11:15.947557 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cf48f3-aa94-43b1-9a1e-25a049d52213 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:15.957948 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cf48f3-aa94-43b1-9a1e-25a049d52213 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:15.964335 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56bbdb1-da63-4e2b-8b22-4f4b9113d0ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:15.965041 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56bbdb1-da63-4e2b-8b22-4f4b9113d0ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:15.965846 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56bbdb1-da63-4e2b-8b22-4f4b9113d0ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:15.966198 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56bbdb1-da63-4e2b-8b22-4f4b9113d0ae admin admin] PATH_INFO: `/v3/projects/8b2025a43dc140dca0e0dd6b81039c70/users/babf85d26f8e4c349192ef7d2c4978d8/roles/56e4a96094e9434cbbafb22579c034f5` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:15.982349 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56bbdb1-da63-4e2b-8b22-4f4b9113d0ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b2025a43dc140dca0e0dd6b81039c70, user_id=babf85d26f8e4c349192ef7d2c4978d8, role_id=56e4a96094e9434cbbafb22579c034f5)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:15.985100 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56bbdb1-da63-4e2b-8b22-4f4b9113d0ae admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:16.093520 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 991/1995] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:11:15 2022] PUT /identity/v3/projects/8b2025a43dc140dca0e0dd6b81039c70/users/babf85d26f8e4c349192ef7d2c4978d8/roles/56e4a96094e9434cbbafb22579c034f5 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:11:16.110662 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fbc959-549a-44a8-840e-ed57e4cda42a None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:16.110662 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fbc959-549a-44a8-840e-ed57e4cda42a None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:16.110662 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fbc959-549a-44a8-840e-ed57e4cda42a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:16.121572 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:16.133821 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-28fbc959-549a-44a8-840e-ed57e4cda42a None None] MFA Rules not processed for user `babf85d26f8e4c349192ef7d2c4978d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:11:16.135059 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-28fbc959-549a-44a8-840e-ed57e4cda42a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:16.217804 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1005/1996] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:11:16 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:11:16.218669 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56bbdb1-da63-4e2b-8b22-4f4b9113d0ae admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:16.310597 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56bbdb1-da63-4e2b-8b22-4f4b9113d0ae admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:16.315770 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeaac34-a26a-4a0d-b859-7b55b1dc3422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:16.316389 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeaac34-a26a-4a0d-b859-7b55b1dc3422 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:16.316703 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeaac34-a26a-4a0d-b859-7b55b1dc3422 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:16.316999 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeaac34-a26a-4a0d-b859-7b55b1dc3422 admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:16.317654 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeaac34-a26a-4a0d-b859-7b55b1dc3422 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:16.319663 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeaac34-a26a-4a0d-b859-7b55b1dc3422 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:16.384492 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:16.424992 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 992/1997] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:11:16 2022] POST /identity/v3/projects => generated 385 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 22:11:16.428967 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fbc959-549a-44a8-840e-ed57e4cda42a None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:16.439654 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fbc959-549a-44a8-840e-ed57e4cda42a None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:16.494450 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f8494e-43e0-45b7-8f46-d69a486ea0d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:16.495051 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f8494e-43e0-45b7-8f46-d69a486ea0d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:16.495335 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f8494e-43e0-45b7-8f46-d69a486ea0d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:16.495603 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f8494e-43e0-45b7-8f46-d69a486ea0d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:16.496232 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f8494e-43e0-45b7-8f46-d69a486ea0d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:16.498984 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f8494e-43e0-45b7-8f46-d69a486ea0d7 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:16.628553 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1006/1998] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:11:16 2022] POST /identity/v3/users => generated 299 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:11:16.629651 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeaac34-a26a-4a0d-b859-7b55b1dc3422 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:16.646096 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeaac34-a26a-4a0d-b859-7b55b1dc3422 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:16.653698 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26feeae-cb2d-4b60-b9da-b94671f32a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:16.654063 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26feeae-cb2d-4b60-b9da-b94671f32a60 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:16.654392 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26feeae-cb2d-4b60-b9da-b94671f32a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:16.654764 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26feeae-cb2d-4b60-b9da-b94671f32a60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:16.655427 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26feeae-cb2d-4b60-b9da-b94671f32a60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:16.657586 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26feeae-cb2d-4b60-b9da-b94671f32a60 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:16.670940 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 993/1999] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:11:16 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 22:11:16.675007 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f8494e-43e0-45b7-8f46-d69a486ea0d7 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:16.686219 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f8494e-43e0-45b7-8f46-d69a486ea0d7 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:16.691166 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ab0421-9ebc-485e-b4bb-1eae30eeff8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:16.691889 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ab0421-9ebc-485e-b4bb-1eae30eeff8c admin admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:16.692183 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ab0421-9ebc-485e-b4bb-1eae30eeff8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:16.692476 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ab0421-9ebc-485e-b4bb-1eae30eeff8c admin admin] PATH_INFO: `/v3/projects/356245ee0b1d442da02e20e5cdd6dcf4/users/e43c25dedc614f1bb53ed0c48ee70701/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:16.709176 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ab0421-9ebc-485e-b4bb-1eae30eeff8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=356245ee0b1d442da02e20e5cdd6dcf4, user_id=e43c25dedc614f1bb53ed0c48ee70701, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:16.712128 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ab0421-9ebc-485e-b4bb-1eae30eeff8c admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:16.803998 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1007/2000] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:11:16 2022] PUT /identity/v3/projects/356245ee0b1d442da02e20e5cdd6dcf4/users/e43c25dedc614f1bb53ed0c48ee70701/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:11:16.807769 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26feeae-cb2d-4b60-b9da-b94671f32a60 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:16.820177 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26feeae-cb2d-4b60-b9da-b94671f32a60 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:16.876472 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c919a2af-d0ae-4d83-ac78-bc5b3f6dcb7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:16.877121 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c919a2af-d0ae-4d83-ac78-bc5b3f6dcb7b admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:16.877284 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c919a2af-d0ae-4d83-ac78-bc5b3f6dcb7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:16.877555 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c919a2af-d0ae-4d83-ac78-bc5b3f6dcb7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:16.878187 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c919a2af-d0ae-4d83-ac78-bc5b3f6dcb7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:16.879774 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c919a2af-d0ae-4d83-ac78-bc5b3f6dcb7b admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:16.890600 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=81,keystone:DELETE=14,keystone:INSERT=5 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:16.890755 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 994/2001] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:11:16 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:11:16.894009 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ab0421-9ebc-485e-b4bb-1eae30eeff8c admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:16.904492 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ab0421-9ebc-485e-b4bb-1eae30eeff8c admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:16.909258 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3700f749-a7af-45ae-aa8a-58d8d5ed5a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:16.909909 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3700f749-a7af-45ae-aa8a-58d8d5ed5a56 admin admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:16.910192 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3700f749-a7af-45ae-aa8a-58d8d5ed5a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:16.910478 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3700f749-a7af-45ae-aa8a-58d8d5ed5a56 admin admin] PATH_INFO: `/v3/projects/356245ee0b1d442da02e20e5cdd6dcf4/users/e43c25dedc614f1bb53ed0c48ee70701/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:16.912378 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3700f749-a7af-45ae-aa8a-58d8d5ed5a56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=356245ee0b1d442da02e20e5cdd6dcf4, user_id=e43c25dedc614f1bb53ed0c48ee70701, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:16.915024 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3700f749-a7af-45ae-aa8a-58d8d5ed5a56 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:16.930232 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1008/2002] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:11:16 2022] PUT /identity/v3/projects/356245ee0b1d442da02e20e5cdd6dcf4/users/e43c25dedc614f1bb53ed0c48ee70701/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:11:16.941879 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c919a2af-d0ae-4d83-ac78-bc5b3f6dcb7b admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:16.943943 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-c919a2af-d0ae-4d83-ac78-bc5b3f6dcb7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:16.946033 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91126519-00af-4250-8955-68e0d9c2b5bb None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:16.946276 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91126519-00af-4250-8955-68e0d9c2b5bb None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:16.946546 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91126519-00af-4250-8955-68e0d9c2b5bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:16.960917 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c919a2af-d0ae-4d83-ac78-bc5b3f6dcb7b admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:16.985239 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-91126519-00af-4250-8955-68e0d9c2b5bb None None] MFA Rules not processed for user `e43c25dedc614f1bb53ed0c48ee70701`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:11:16.987520 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-91126519-00af-4250-8955-68e0d9c2b5bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:17.051616 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d545fc1-64ed-4740-9c1f-ff9e22d9c7ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:17.052485 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d545fc1-64ed-4740-9c1f-ff9e22d9c7ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:17.052870 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d545fc1-64ed-4740-9c1f-ff9e22d9c7ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:17.053291 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d545fc1-64ed-4740-9c1f-ff9e22d9c7ca admin admin] PATH_INFO: `/v3/users/f7c9848c541042f2a5a320638429fb4a` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:17.054619 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d545fc1-64ed-4740-9c1f-ff9e22d9c7ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7c9848c541042f2a5a320638429fb4a)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:17.056801 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d545fc1-64ed-4740-9c1f-ff9e22d9c7ca admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:17.078274 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1009/2003] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:11:16 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:11:17.088974 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91126519-00af-4250-8955-68e0d9c2b5bb None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:17.090459 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-91126519-00af-4250-8955-68e0d9c2b5bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:17.148280 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91126519-00af-4250-8955-68e0d9c2b5bb None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:17.152363 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb5c1a8-d8da-4535-a45a-cd04f9c6b892 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa71c71de263478da234f3c5716a6e5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:17.153061 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5c1a8-d8da-4535-a45a-cd04f9c6b892 service cinder] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:17.153454 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5c1a8-d8da-4535-a45a-cd04f9c6b892 service cinder] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:17.153729 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5c1a8-d8da-4535-a45a-cd04f9c6b892 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:17.163492 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb5c1a8-d8da-4535-a45a-cd04f9c6b892 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:17.165048 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb5c1a8-d8da-4535-a45a-cd04f9c6b892 service cinder] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:17.175452 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1010/2004] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:11:17 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 22:11:17.298172 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-1d545fc1-64ed-4740-9c1f-ff9e22d9c7ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7c9848c541042f2a5a320638429fb4a'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:17.312066 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-1d545fc1-64ed-4740-9c1f-ff9e22d9c7ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7c9848c541042f2a5a320638429fb4a'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:17.315527 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:DELETE=3,keystone:SELECT=92,keystone:INSERT=5 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:17.323633 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-1d545fc1-64ed-4740-9c1f-ff9e22d9c7ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7c9848c541042f2a5a320638429fb4a'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:17.324280 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-1d545fc1-64ed-4740-9c1f-ff9e22d9c7ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7c9848c541042f2a5a320638429fb4a'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:17.325839 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-1d545fc1-64ed-4740-9c1f-ff9e22d9c7ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7c9848c541042f2a5a320638429fb4a'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:17.476219 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 995/2005] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:11:16 2022] DELETE /identity/v3/users/f7c9848c541042f2a5a320638429fb4a => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 22:11:17.480774 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187860d1-6634-4b69-9e37-e5b98255c51a None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:17.481740 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187860d1-6634-4b69-9e37-e5b98255c51a None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:17.482017 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187860d1-6634-4b69-9e37-e5b98255c51a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:17.501457 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-187860d1-6634-4b69-9e37-e5b98255c51a None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:11:17.502998 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-187860d1-6634-4b69-9e37-e5b98255c51a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:17.570865 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1011/2006] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:11:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:11:17.583702 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d545fc1-64ed-4740-9c1f-ff9e22d9c7ca admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:17.585283 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-1d545fc1-64ed-4740-9c1f-ff9e22d9c7ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:17.650004 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d545fc1-64ed-4740-9c1f-ff9e22d9c7ca admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:17.650881 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607cd9fd-43b8-419b-8218-5ed8a2f61ecb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:17.651503 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607cd9fd-43b8-419b-8218-5ed8a2f61ecb service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:17.651832 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607cd9fd-43b8-419b-8218-5ed8a2f61ecb service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:17.652131 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607cd9fd-43b8-419b-8218-5ed8a2f61ecb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:17.663354 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607cd9fd-43b8-419b-8218-5ed8a2f61ecb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:17.664872 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607cd9fd-43b8-419b-8218-5ed8a2f61ecb service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:17.675590 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 996/2007] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:11:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:11:17.910637 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:18.237332 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187860d1-6634-4b69-9e37-e5b98255c51a None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:18.238855 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-187860d1-6634-4b69-9e37-e5b98255c51a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:18.252192 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187860d1-6634-4b69-9e37-e5b98255c51a None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:18.257730 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c60edc2-1101-4f76-9be6-176e78fe3966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:18.258375 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c60edc2-1101-4f76-9be6-176e78fe3966 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:18.258656 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c60edc2-1101-4f76-9be6-176e78fe3966 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:18.259010 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c60edc2-1101-4f76-9be6-176e78fe3966 admin admin] PATH_INFO: `/v3/projects/1379403844b2452ba2252913f90a6d8d` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:18.260426 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c60edc2-1101-4f76-9be6-176e78fe3966 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1379403844b2452ba2252913f90a6d8d)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:18.262641 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c60edc2-1101-4f76-9be6-176e78fe3966 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:18.421014 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-8c60edc2-1101-4f76-9be6-176e78fe3966 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1379403844b2452ba2252913f90a6d8d', 'request_id': 'req-8c60edc2-1101-4f76-9be6-176e78fe3966'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:18.515365 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-8c60edc2-1101-4f76-9be6-176e78fe3966 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1379403844b2452ba2252913f90a6d8d', 'request_id': 'req-8c60edc2-1101-4f76-9be6-176e78fe3966'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:18.527411 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-8c60edc2-1101-4f76-9be6-176e78fe3966 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1379403844b2452ba2252913f90a6d8d', 'request_id': 'req-8c60edc2-1101-4f76-9be6-176e78fe3966'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:18.538204 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-8c60edc2-1101-4f76-9be6-176e78fe3966 admin admin] The token cache is being invalidate because project 1379403844b2452ba2252913f90a6d8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:11:18.538495 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-8c60edc2-1101-4f76-9be6-176e78fe3966 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:18.539157 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-8c60edc2-1101-4f76-9be6-176e78fe3966 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:18.540906 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1012/2008] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:11:18 2022] DELETE /identity/v3/projects/1379403844b2452ba2252913f90a6d8d => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:11:18.545090 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607cd9fd-43b8-419b-8218-5ed8a2f61ecb service neutron] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:18.554666 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607cd9fd-43b8-419b-8218-5ed8a2f61ecb service neutron] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:18.618488 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1b7303-7ee8-4b1f-8883-f60b3b7047fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:18.619124 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1b7303-7ee8-4b1f-8883-f60b3b7047fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:18.619403 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1b7303-7ee8-4b1f-8883-f60b3b7047fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:18.619677 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1b7303-7ee8-4b1f-8883-f60b3b7047fd admin admin] PATH_INFO: `/v3/users/c8fa8f6e617a46869563267c2451fb17` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:18.621053 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1b7303-7ee8-4b1f-8883-f60b3b7047fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8fa8f6e617a46869563267c2451fb17)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:18.626368 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1b7303-7ee8-4b1f-8883-f60b3b7047fd admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:18.626512 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c60edc2-1101-4f76-9be6-176e78fe3966 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:18.628010 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-8c60edc2-1101-4f76-9be6-176e78fe3966 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:18.645948 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c60edc2-1101-4f76-9be6-176e78fe3966 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:18.652087 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6450a611-6a94-4c6d-bde5-f75d65edcc75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:18.653437 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6450a611-6a94-4c6d-bde5-f75d65edcc75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:18.653993 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6450a611-6a94-4c6d-bde5-f75d65edcc75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:18.654449 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6450a611-6a94-4c6d-bde5-f75d65edcc75 admin admin] PATH_INFO: `/v3/users/babf85d26f8e4c349192ef7d2c4978d8` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:18.656017 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6450a611-6a94-4c6d-bde5-f75d65edcc75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=babf85d26f8e4c349192ef7d2c4978d8)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:18.658179 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6450a611-6a94-4c6d-bde5-f75d65edcc75 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:18.900093 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-1d1b7303-7ee8-4b1f-8883-f60b3b7047fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8fa8f6e617a46869563267c2451fb17'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:18.912730 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-1d1b7303-7ee8-4b1f-8883-f60b3b7047fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8fa8f6e617a46869563267c2451fb17'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:18.920145 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:18.927632 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-1d1b7303-7ee8-4b1f-8883-f60b3b7047fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8fa8f6e617a46869563267c2451fb17'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:18.929047 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-1d1b7303-7ee8-4b1f-8883-f60b3b7047fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8fa8f6e617a46869563267c2451fb17'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:18.929669 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-1d1b7303-7ee8-4b1f-8883-f60b3b7047fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8fa8f6e617a46869563267c2451fb17'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:18.938329 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-6450a611-6a94-4c6d-bde5-f75d65edcc75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'babf85d26f8e4c349192ef7d2c4978d8'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:18.953417 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-6450a611-6a94-4c6d-bde5-f75d65edcc75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'babf85d26f8e4c349192ef7d2c4978d8'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:19.087373 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 997/2009] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:11:18 2022] DELETE /identity/v3/users/c8fa8f6e617a46869563267c2451fb17 => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:11:19.161297 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-6450a611-6a94-4c6d-bde5-f75d65edcc75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'babf85d26f8e4c349192ef7d2c4978d8'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:19.169824 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-6450a611-6a94-4c6d-bde5-f75d65edcc75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'babf85d26f8e4c349192ef7d2c4978d8'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:19.170544 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-6450a611-6a94-4c6d-bde5-f75d65edcc75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'babf85d26f8e4c349192ef7d2c4978d8'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:19.177354 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1013/2010] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:11:18 2022] DELETE /identity/v3/users/babf85d26f8e4c349192ef7d2c4978d8 => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 22:11:19.181729 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d6e26-f3f6-4f12-96d7-29a9e0cc1d57 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:19.183122 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d6e26-f3f6-4f12-96d7-29a9e0cc1d57 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:19.183804 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d6e26-f3f6-4f12-96d7-29a9e0cc1d57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:19.207211 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-4a0d6e26-f3f6-4f12-96d7-29a9e0cc1d57 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:11:19.208519 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-4a0d6e26-f3f6-4f12-96d7-29a9e0cc1d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:19.286029 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 998/2011] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:11:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:11:19.295770 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6450a611-6a94-4c6d-bde5-f75d65edcc75 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:19.297275 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-6450a611-6a94-4c6d-bde5-f75d65edcc75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:19.353961 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6450a611-6a94-4c6d-bde5-f75d65edcc75 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:19.357995 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e83a6f-03fd-4265-b297-f23c19e7c777 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:19.358611 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e83a6f-03fd-4265-b297-f23c19e7c777 service neutron] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:19.358915 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e83a6f-03fd-4265-b297-f23c19e7c777 service neutron] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:19.359246 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e83a6f-03fd-4265-b297-f23c19e7c777 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:19.368938 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e83a6f-03fd-4265-b297-f23c19e7c777 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:19.370464 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e83a6f-03fd-4265-b297-f23c19e7c777 service neutron] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:19.380934 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1014/2012] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:11:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:11:19.686506 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0d6e26-f3f6-4f12-96d7-29a9e0cc1d57 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:19.688125 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-4a0d6e26-f3f6-4f12-96d7-29a9e0cc1d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:19.701708 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0d6e26-f3f6-4f12-96d7-29a9e0cc1d57 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:19.709712 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52431e07-dfd9-4362-af2c-ff4196c4e175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:19.710659 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52431e07-dfd9-4362-af2c-ff4196c4e175 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:19.711358 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52431e07-dfd9-4362-af2c-ff4196c4e175 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:19.712772 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:19.713351 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52431e07-dfd9-4362-af2c-ff4196c4e175 admin admin] PATH_INFO: `/v3/projects/508eae5d8f214f31b68fced6a56140d8` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:19.716603 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52431e07-dfd9-4362-af2c-ff4196c4e175 admin admin] RBAC: Authorizing `identity:delete_project(project_id=508eae5d8f214f31b68fced6a56140d8)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:19.732425 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52431e07-dfd9-4362-af2c-ff4196c4e175 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:19.957714 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e83a6f-03fd-4265-b297-f23c19e7c777 service neutron] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:19.959746 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-b3e83a6f-03fd-4265-b297-f23c19e7c777 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:19.971306 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-52431e07-dfd9-4362-af2c-ff4196c4e175 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '508eae5d8f214f31b68fced6a56140d8', 'request_id': 'req-52431e07-dfd9-4362-af2c-ff4196c4e175'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:19.984277 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e83a6f-03fd-4265-b297-f23c19e7c777 service neutron] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:20.041765 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb217a4e-a72f-4741-9ec7-04b88cc8bc28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:20.042399 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb217a4e-a72f-4741-9ec7-04b88cc8bc28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:20.042684 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb217a4e-a72f-4741-9ec7-04b88cc8bc28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:20.042957 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb217a4e-a72f-4741-9ec7-04b88cc8bc28 admin admin] PATH_INFO: `/v3/projects/8b2025a43dc140dca0e0dd6b81039c70` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:20.045013 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb217a4e-a72f-4741-9ec7-04b88cc8bc28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b2025a43dc140dca0e0dd6b81039c70)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:20.046343 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-52431e07-dfd9-4362-af2c-ff4196c4e175 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '508eae5d8f214f31b68fced6a56140d8', 'request_id': 'req-52431e07-dfd9-4362-af2c-ff4196c4e175'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:20.046914 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb217a4e-a72f-4741-9ec7-04b88cc8bc28 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:20.052583 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-52431e07-dfd9-4362-af2c-ff4196c4e175 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '508eae5d8f214f31b68fced6a56140d8', 'request_id': 'req-52431e07-dfd9-4362-af2c-ff4196c4e175'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:20.064594 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-52431e07-dfd9-4362-af2c-ff4196c4e175 admin admin] The token cache is being invalidate because project 508eae5d8f214f31b68fced6a56140d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:11:20.064829 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-52431e07-dfd9-4362-af2c-ff4196c4e175 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:20.065506 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-52431e07-dfd9-4362-af2c-ff4196c4e175 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:20.067010 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 999/2013] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:11:19 2022] DELETE /identity/v3/projects/508eae5d8f214f31b68fced6a56140d8 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:11:20.071815 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec0e490-db5f-4adb-b7aa-856e6eb9ce53 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:20.072170 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec0e490-db5f-4adb-b7aa-856e6eb9ce53 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:20.072473 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec0e490-db5f-4adb-b7aa-856e6eb9ce53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:20.091794 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-7ec0e490-db5f-4adb-b7aa-856e6eb9ce53 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:11:20.093061 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-7ec0e490-db5f-4adb-b7aa-856e6eb9ce53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:20.101304 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1000/2014] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:11:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 22:11:20.110731 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753a01e7-5697-4613-84e8-f35b7db2319c None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:20.111014 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753a01e7-5697-4613-84e8-f35b7db2319c None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:20.111295 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753a01e7-5697-4613-84e8-f35b7db2319c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:20.131288 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-753a01e7-5697-4613-84e8-f35b7db2319c None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:11:20.132504 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-753a01e7-5697-4613-84e8-f35b7db2319c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:20.140726 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1001/2015] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:11:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 22:11:20.144926 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753a01e7-5697-4613-84e8-f35b7db2319c None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:20.153876 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753a01e7-5697-4613-84e8-f35b7db2319c None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:20.160716 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-eb217a4e-a72f-4741-9ec7-04b88cc8bc28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b2025a43dc140dca0e0dd6b81039c70', 'request_id': 'req-eb217a4e-a72f-4741-9ec7-04b88cc8bc28'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:20.172291 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:20.210531 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3633c059-0957-4150-9177-3ac3e062b318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:20.211233 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3633c059-0957-4150-9177-3ac3e062b318 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:20.211556 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3633c059-0957-4150-9177-3ac3e062b318 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:20.211834 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3633c059-0957-4150-9177-3ac3e062b318 admin admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:20.212509 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3633c059-0957-4150-9177-3ac3e062b318 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 22:11:20.215731 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3633c059-0957-4150-9177-3ac3e062b318 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:20.215731 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3633c059-0957-4150-9177-3ac3e062b318 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:20.227048 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1002/2016] 10.222.0.18 () {58 vars in 1153 bytes} [Sat Aug 6 22:11:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 22:11:20.231856 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3633c059-0957-4150-9177-3ac3e062b318 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:20.235228 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-eb217a4e-a72f-4741-9ec7-04b88cc8bc28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b2025a43dc140dca0e0dd6b81039c70', 'request_id': 'req-eb217a4e-a72f-4741-9ec7-04b88cc8bc28'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:20.241564 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3633c059-0957-4150-9177-3ac3e062b318 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:20.250268 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-eb217a4e-a72f-4741-9ec7-04b88cc8bc28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b2025a43dc140dca0e0dd6b81039c70', 'request_id': 'req-eb217a4e-a72f-4741-9ec7-04b88cc8bc28'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:20.250410 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0a1c59-84d5-40c1-9e32-71a363c06e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:20.250410 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a1c59-84d5-40c1-9e32-71a363c06e64 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:20.250410 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a1c59-84d5-40c1-9e32-71a363c06e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:20.250410 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a1c59-84d5-40c1-9e32-71a363c06e64 admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:20.250707 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0a1c59-84d5-40c1-9e32-71a363c06e64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:20.250707 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0a1c59-84d5-40c1-9e32-71a363c06e64 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:20.253433 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-eb217a4e-a72f-4741-9ec7-04b88cc8bc28 admin admin] The token cache is being invalidate because project 8b2025a43dc140dca0e0dd6b81039c70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:11:20.253724 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-eb217a4e-a72f-4741-9ec7-04b88cc8bc28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:20.254401 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-eb217a4e-a72f-4741-9ec7-04b88cc8bc28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:20.255992 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1015/2017] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:11:19 2022] DELETE /identity/v3/projects/8b2025a43dc140dca0e0dd6b81039c70 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 22:11:20.259642 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb217a4e-a72f-4741-9ec7-04b88cc8bc28 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:20.261066 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-eb217a4e-a72f-4741-9ec7-04b88cc8bc28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:20.275340 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb217a4e-a72f-4741-9ec7-04b88cc8bc28 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:20.280587 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db079add-eb78-4ba9-bcb9-351fd1082fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:20.281221 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db079add-eb78-4ba9-bcb9-351fd1082fde admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:20.281525 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db079add-eb78-4ba9-bcb9-351fd1082fde admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:20.281795 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db079add-eb78-4ba9-bcb9-351fd1082fde admin admin] PATH_INFO: `/v3/users/e43c25dedc614f1bb53ed0c48ee70701` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:20.282915 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db079add-eb78-4ba9-bcb9-351fd1082fde admin admin] RBAC: Authorizing `identity:delete_user(user_id=e43c25dedc614f1bb53ed0c48ee70701)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:20.284885 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db079add-eb78-4ba9-bcb9-351fd1082fde admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:20.333643 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1003/2018] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:11:20 2022] POST /identity/v3/projects => generated 383 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 22:11:20.337889 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0a1c59-84d5-40c1-9e32-71a363c06e64 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:20.339570 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-ce0a1c59-84d5-40c1-9e32-71a363c06e64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:20.411764 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0a1c59-84d5-40c1-9e32-71a363c06e64 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:20.416397 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac440598-8ead-4a4a-b53d-396c5da6746f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:20.417112 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac440598-8ead-4a4a-b53d-396c5da6746f admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:20.417390 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac440598-8ead-4a4a-b53d-396c5da6746f admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:20.417661 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac440598-8ead-4a4a-b53d-396c5da6746f admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:20.418886 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac440598-8ead-4a4a-b53d-396c5da6746f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:20.421013 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac440598-8ead-4a4a-b53d-396c5da6746f admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:20.493568 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-db079add-eb78-4ba9-bcb9-351fd1082fde admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e43c25dedc614f1bb53ed0c48ee70701'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:20.505337 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-db079add-eb78-4ba9-bcb9-351fd1082fde admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e43c25dedc614f1bb53ed0c48ee70701'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:20.515292 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1004/2019] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:11:20 2022] POST /identity/v3/users => generated 292 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 22:11:20.519477 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac440598-8ead-4a4a-b53d-396c5da6746f admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:20.529347 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac440598-8ead-4a4a-b53d-396c5da6746f admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:20.533899 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3453e4be-dbb5-4bf6-934d-3627a8e72482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:20.534475 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3453e4be-dbb5-4bf6-934d-3627a8e72482 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:20.534752 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3453e4be-dbb5-4bf6-934d-3627a8e72482 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:20.535020 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3453e4be-dbb5-4bf6-934d-3627a8e72482 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:20.535640 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3453e4be-dbb5-4bf6-934d-3627a8e72482 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:20.537301 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3453e4be-dbb5-4bf6-934d-3627a8e72482 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:20.547404 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1005/2020] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:11:20 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 22:11:20.551297 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3453e4be-dbb5-4bf6-934d-3627a8e72482 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:20.560542 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3453e4be-dbb5-4bf6-934d-3627a8e72482 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:20.565256 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e6c164-7b10-4c1c-9a19-fbce8cfa0620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:20.565787 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e6c164-7b10-4c1c-9a19-fbce8cfa0620 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:20.566062 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e6c164-7b10-4c1c-9a19-fbce8cfa0620 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:20.566330 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e6c164-7b10-4c1c-9a19-fbce8cfa0620 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:20.566925 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e6c164-7b10-4c1c-9a19-fbce8cfa0620 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:20.568476 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e6c164-7b10-4c1c-9a19-fbce8cfa0620 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:20.579918 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1006/2021] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:11:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 22:11:20.596240 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e6c164-7b10-4c1c-9a19-fbce8cfa0620 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:20.606202 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e6c164-7b10-4c1c-9a19-fbce8cfa0620 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:20.610972 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8713bc-0515-4694-9f77-1bc264cefad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:20.611628 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8713bc-0515-4694-9f77-1bc264cefad2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:20.612001 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8713bc-0515-4694-9f77-1bc264cefad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:20.612172 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8713bc-0515-4694-9f77-1bc264cefad2 admin admin] PATH_INFO: `/v3/projects/9ea3017518bb4947b33d7ee8d8ffd952/users/653fb6b57de4454da7e85479124e64b1/roles/56e4a96094e9434cbbafb22579c034f5` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:20.627927 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8713bc-0515-4694-9f77-1bc264cefad2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ea3017518bb4947b33d7ee8d8ffd952, user_id=653fb6b57de4454da7e85479124e64b1, role_id=56e4a96094e9434cbbafb22579c034f5)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:20.630724 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8713bc-0515-4694-9f77-1bc264cefad2 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:20.652354 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-db079add-eb78-4ba9-bcb9-351fd1082fde admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e43c25dedc614f1bb53ed0c48ee70701'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:20.660507 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-db079add-eb78-4ba9-bcb9-351fd1082fde admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e43c25dedc614f1bb53ed0c48ee70701'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:20.661140 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-db079add-eb78-4ba9-bcb9-351fd1082fde admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e43c25dedc614f1bb53ed0c48ee70701'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:20.667050 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1016/2022] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:11:20 2022] DELETE /identity/v3/users/e43c25dedc614f1bb53ed0c48ee70701 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 22:11:20.727361 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1007/2023] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:11:20 2022] PUT /identity/v3/projects/9ea3017518bb4947b33d7ee8d8ffd952/users/653fb6b57de4454da7e85479124e64b1/roles/56e4a96094e9434cbbafb22579c034f5 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 22:11:20.737655 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9f7c3f-cdf7-4891-9f32-4a07a83cec63 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:20.737941 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9f7c3f-cdf7-4891-9f32-4a07a83cec63 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:20.738228 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9f7c3f-cdf7-4891-9f32-4a07a83cec63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:20.743780 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:20.767359 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-4b9f7c3f-cdf7-4891-9f32-4a07a83cec63 None None] MFA Rules not processed for user `653fb6b57de4454da7e85479124e64b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:11:20.768623 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-4b9f7c3f-cdf7-4891-9f32-4a07a83cec63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:20.836880 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1017/2024] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:11:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:11:20.842143 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8713bc-0515-4694-9f77-1bc264cefad2 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:20.843916 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-8f8713bc-0515-4694-9f77-1bc264cefad2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:20.940011 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8713bc-0515-4694-9f77-1bc264cefad2 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:20.951359 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9244e693-0774-4064-bc5d-fb99fef05a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:20.951359 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9244e693-0774-4064-bc5d-fb99fef05a2d admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:20.951359 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9244e693-0774-4064-bc5d-fb99fef05a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:20.951359 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9244e693-0774-4064-bc5d-fb99fef05a2d admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:20.956981 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9244e693-0774-4064-bc5d-fb99fef05a2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:20.964819 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9244e693-0774-4064-bc5d-fb99fef05a2d admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:21.013461 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:21.077404 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1008/2025] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:11:20 2022] POST /identity/v3/projects => generated 379 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:11:21.081381 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9f7c3f-cdf7-4891-9f32-4a07a83cec63 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:21.092372 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9f7c3f-cdf7-4891-9f32-4a07a83cec63 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:21.107430 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9244e693-0774-4064-bc5d-fb99fef05a2d admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:21.109220 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-9244e693-0774-4064-bc5d-fb99fef05a2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:21.179555 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caeaec2-e3aa-40fb-b996-02b7ef93c039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:21.180271 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caeaec2-e3aa-40fb-b996-02b7ef93c039 admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:21.180634 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caeaec2-e3aa-40fb-b996-02b7ef93c039 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:21.180970 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caeaec2-e3aa-40fb-b996-02b7ef93c039 admin admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:21.181688 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caeaec2-e3aa-40fb-b996-02b7ef93c039 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:21.185593 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caeaec2-e3aa-40fb-b996-02b7ef93c039 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:21.192471 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9244e693-0774-4064-bc5d-fb99fef05a2d admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:21.197559 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a99a28-2d7b-4971-804d-fe51743cfa63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:21.198253 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a99a28-2d7b-4971-804d-fe51743cfa63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:21.198573 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a99a28-2d7b-4971-804d-fe51743cfa63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:21.198893 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a99a28-2d7b-4971-804d-fe51743cfa63 admin admin] PATH_INFO: `/v3/projects/356245ee0b1d442da02e20e5cdd6dcf4` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:21.200390 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a99a28-2d7b-4971-804d-fe51743cfa63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=356245ee0b1d442da02e20e5cdd6dcf4)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:21.202472 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a99a28-2d7b-4971-804d-fe51743cfa63 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:21.309576 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1018/2026] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:11:21 2022] POST /identity/v3/users => generated 296 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:11:21.313498 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caeaec2-e3aa-40fb-b996-02b7ef93c039 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:21.323007 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caeaec2-e3aa-40fb-b996-02b7ef93c039 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:21.328833 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f0407f-f121-4a27-80a1-77fb09a31a77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:21.330076 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f0407f-f121-4a27-80a1-77fb09a31a77 admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:21.332220 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f0407f-f121-4a27-80a1-77fb09a31a77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:21.332220 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f0407f-f121-4a27-80a1-77fb09a31a77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:21.332533 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f0407f-f121-4a27-80a1-77fb09a31a77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:21.334748 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f0407f-f121-4a27-80a1-77fb09a31a77 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:21.345812 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1019/2027] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:11:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 22:11:21.349228 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-c1a99a28-2d7b-4971-804d-fe51743cfa63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '356245ee0b1d442da02e20e5cdd6dcf4', 'request_id': 'req-c1a99a28-2d7b-4971-804d-fe51743cfa63'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:21.352115 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f0407f-f121-4a27-80a1-77fb09a31a77 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:21.363235 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f0407f-f121-4a27-80a1-77fb09a31a77 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:21.412025 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-c1a99a28-2d7b-4971-804d-fe51743cfa63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '356245ee0b1d442da02e20e5cdd6dcf4', 'request_id': 'req-c1a99a28-2d7b-4971-804d-fe51743cfa63'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:21.418410 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-c1a99a28-2d7b-4971-804d-fe51743cfa63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '356245ee0b1d442da02e20e5cdd6dcf4', 'request_id': 'req-c1a99a28-2d7b-4971-804d-fe51743cfa63'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:21.421738 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c1eb30-1b29-4846-be60-c91a8a8531fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:21.422394 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c1eb30-1b29-4846-be60-c91a8a8531fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:21.422674 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c1eb30-1b29-4846-be60-c91a8a8531fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:21.422941 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c1eb30-1b29-4846-be60-c91a8a8531fe admin admin] PATH_INFO: `/v3/projects/4a4e40c23f8544e3a65bb78c085822a7/users/e31371fe75084e6596c66995bde3cce7/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:21.430441 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-c1a99a28-2d7b-4971-804d-fe51743cfa63 admin admin] The token cache is being invalidate because project 356245ee0b1d442da02e20e5cdd6dcf4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:11:21.430736 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-c1a99a28-2d7b-4971-804d-fe51743cfa63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:21.431394 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-c1a99a28-2d7b-4971-804d-fe51743cfa63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:21.433558 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1009/2028] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:11:21 2022] DELETE /identity/v3/projects/356245ee0b1d442da02e20e5cdd6dcf4 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 22:11:21.437381 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c1eb30-1b29-4846-be60-c91a8a8531fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a4e40c23f8544e3a65bb78c085822a7, user_id=e31371fe75084e6596c66995bde3cce7, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:21.437808 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e534b3a-80b3-4570-8a57-15253c3f5cf4 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:21.438093 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e534b3a-80b3-4570-8a57-15253c3f5cf4 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:21.438368 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e534b3a-80b3-4570-8a57-15253c3f5cf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:21.440075 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c1eb30-1b29-4846-be60-c91a8a8531fe admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:21.458108 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-7e534b3a-80b3-4570-8a57-15253c3f5cf4 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:11:21.459377 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-7e534b3a-80b3-4570-8a57-15253c3f5cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:21.468279 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1010/2029] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:11:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:11:21.478980 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a04a21-0801-44e5-960a-53baf69bd489 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:21.479271 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a04a21-0801-44e5-960a-53baf69bd489 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:21.479658 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a04a21-0801-44e5-960a-53baf69bd489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:21.503750 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-40a04a21-0801-44e5-960a-53baf69bd489 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:11:21.509413 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-40a04a21-0801-44e5-960a-53baf69bd489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:21.518073 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1011/2030] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:11:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 22:11:21.519791 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a04a21-0801-44e5-960a-53baf69bd489 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:21.530611 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a04a21-0801-44e5-960a-53baf69bd489 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:21.537306 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1020/2031] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:11:21 2022] PUT /identity/v3/projects/4a4e40c23f8544e3a65bb78c085822a7/users/e31371fe75084e6596c66995bde3cce7/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:11:21.547256 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c1eb30-1b29-4846-be60-c91a8a8531fe admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:21.552774 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-45c1eb30-1b29-4846-be60-c91a8a8531fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:21.591967 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e3ed12-0f65-45fb-a0e7-481b45c589fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:21.592731 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e3ed12-0f65-45fb-a0e7-481b45c589fc admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:21.593088 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e3ed12-0f65-45fb-a0e7-481b45c589fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:21.593418 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e3ed12-0f65-45fb-a0e7-481b45c589fc admin admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:21.594071 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e3ed12-0f65-45fb-a0e7-481b45c589fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 22:11:21.594379 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e3ed12-0f65-45fb-a0e7-481b45c589fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:21.596026 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e3ed12-0f65-45fb-a0e7-481b45c589fc admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:21.609751 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1012/2032] 10.222.0.18 () {58 vars in 1153 bytes} [Sat Aug 6 22:11:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 22:11:21.614117 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e3ed12-0f65-45fb-a0e7-481b45c589fc admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:21.624242 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e3ed12-0f65-45fb-a0e7-481b45c589fc admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:21.628937 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb47377-4fdc-4c8f-b46d-5b054508fc5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:21.629528 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb47377-4fdc-4c8f-b46d-5b054508fc5b admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:21.629908 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb47377-4fdc-4c8f-b46d-5b054508fc5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:21.630074 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb47377-4fdc-4c8f-b46d-5b054508fc5b admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:21.630685 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb47377-4fdc-4c8f-b46d-5b054508fc5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:21.632267 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c1eb30-1b29-4846-be60-c91a8a8531fe admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:21.632852 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb47377-4fdc-4c8f-b46d-5b054508fc5b admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:21.638444 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce50abf-5d0e-4837-b197-12d13483f845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:21.639447 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce50abf-5d0e-4837-b197-12d13483f845 admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:21.639801 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce50abf-5d0e-4837-b197-12d13483f845 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:21.640145 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce50abf-5d0e-4837-b197-12d13483f845 admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:21.640924 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce50abf-5d0e-4837-b197-12d13483f845 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:21.642632 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce50abf-5d0e-4837-b197-12d13483f845 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:21.651935 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:21.654084 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1021/2033] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:11:21 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:11:21.658144 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce50abf-5d0e-4837-b197-12d13483f845 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:21.667950 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce50abf-5d0e-4837-b197-12d13483f845 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:21.672604 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425687c3-b135-45da-b821-30fc2a01db29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:21.673228 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425687c3-b135-45da-b821-30fc2a01db29 admin admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:21.673504 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425687c3-b135-45da-b821-30fc2a01db29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:21.673773 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425687c3-b135-45da-b821-30fc2a01db29 admin admin] PATH_INFO: `/v3/projects/4a4e40c23f8544e3a65bb78c085822a7/users/e31371fe75084e6596c66995bde3cce7/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:21.675683 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425687c3-b135-45da-b821-30fc2a01db29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a4e40c23f8544e3a65bb78c085822a7, user_id=e31371fe75084e6596c66995bde3cce7, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:21.678342 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425687c3-b135-45da-b821-30fc2a01db29 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:21.688197 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1022/2034] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:11:21 2022] PUT /identity/v3/projects/4a4e40c23f8544e3a65bb78c085822a7/users/e31371fe75084e6596c66995bde3cce7/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 30 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 22:11:21.697926 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b731570-8e4f-4e9d-a059-a9b53def3263 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:21.698211 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b731570-8e4f-4e9d-a059-a9b53def3263 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:21.698484 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b731570-8e4f-4e9d-a059-a9b53def3263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:21.726852 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-1b731570-8e4f-4e9d-a059-a9b53def3263 None None] MFA Rules not processed for user `e31371fe75084e6596c66995bde3cce7`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:11:21.728053 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-1b731570-8e4f-4e9d-a059-a9b53def3263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:21.733948 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1013/2035] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:11:21 2022] POST /identity/v3/projects => generated 375 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 22:11:21.738165 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb47377-4fdc-4c8f-b46d-5b054508fc5b admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:21.749771 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:21.802814 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb47377-4fdc-4c8f-b46d-5b054508fc5b admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:21.804795 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1023/2036] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:11:21 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 22:11:21.807817 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f882c8-c399-4b4b-9747-95d67c0bec0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:21.808611 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f882c8-c399-4b4b-9747-95d67c0bec0c admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:21.809583 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f882c8-c399-4b4b-9747-95d67c0bec0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:21.809992 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f882c8-c399-4b4b-9747-95d67c0bec0c admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:21.810737 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f882c8-c399-4b4b-9747-95d67c0bec0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:21.812797 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f882c8-c399-4b4b-9747-95d67c0bec0c admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:21.817595 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b731570-8e4f-4e9d-a059-a9b53def3263 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:21.820161 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-1b731570-8e4f-4e9d-a059-a9b53def3263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:21.950021 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1014/2037] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:11:21 2022] POST /identity/v3/users => generated 288 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 22:11:21.954362 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f882c8-c399-4b4b-9747-95d67c0bec0c admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:21.967729 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f882c8-c399-4b4b-9747-95d67c0bec0c admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:21.973036 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fad573-fabe-45ba-a6ba-f71d6f065e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:21.973745 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fad573-fabe-45ba-a6ba-f71d6f065e59 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:21.974056 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fad573-fabe-45ba-a6ba-f71d6f065e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:21.974293 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fad573-fabe-45ba-a6ba-f71d6f065e59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:21.974990 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fad573-fabe-45ba-a6ba-f71d6f065e59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:21.976918 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fad573-fabe-45ba-a6ba-f71d6f065e59 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:21.988804 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1015/2038] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:11:21 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 22:11:21.995427 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fad573-fabe-45ba-a6ba-f71d6f065e59 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:22.006426 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fad573-fabe-45ba-a6ba-f71d6f065e59 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:22.011578 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094c966d-7207-4bde-bbe3-7da549168726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:22.012020 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b731570-8e4f-4e9d-a059-a9b53def3263 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:22.023530 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c966d-7207-4bde-bbe3-7da549168726 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:22.033891 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c966d-7207-4bde-bbe3-7da549168726 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:22.033891 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c966d-7207-4bde-bbe3-7da549168726 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:22.033891 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094c966d-7207-4bde-bbe3-7da549168726 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:22.033891 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094c966d-7207-4bde-bbe3-7da549168726 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:22.046808 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1016/2039] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:11:21 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 22:11:22.051119 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094c966d-7207-4bde-bbe3-7da549168726 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:22.072477 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094c966d-7207-4bde-bbe3-7da549168726 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:22.075277 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e02549-4c2d-4bb1-bb99-ee349a52cd12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa71c71de263478da234f3c5716a6e5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:22.076020 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e02549-4c2d-4bb1-bb99-ee349a52cd12 service cinder] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:22.076304 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e02549-4c2d-4bb1-bb99-ee349a52cd12 service cinder] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:22.076620 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e02549-4c2d-4bb1-bb99-ee349a52cd12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:22.078387 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbd577b-ec6c-4a18-a85b-c11eaa50b611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:22.079135 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbd577b-ec6c-4a18-a85b-c11eaa50b611 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:22.079484 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbd577b-ec6c-4a18-a85b-c11eaa50b611 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:22.079849 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbd577b-ec6c-4a18-a85b-c11eaa50b611 admin admin] PATH_INFO: `/v3/projects/0d7cfb6fbdb94c16930bdf47ce8f8f47/users/b3873412e4fa4f5aae244082c3a196d4/roles/56e4a96094e9434cbbafb22579c034f5` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:22.096261 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbd577b-ec6c-4a18-a85b-c11eaa50b611 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d7cfb6fbdb94c16930bdf47ce8f8f47, user_id=b3873412e4fa4f5aae244082c3a196d4, role_id=56e4a96094e9434cbbafb22579c034f5)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:22.099108 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbd577b-ec6c-4a18-a85b-c11eaa50b611 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:22.143409 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e02549-4c2d-4bb1-bb99-ee349a52cd12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:22.144887 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e02549-4c2d-4bb1-bb99-ee349a52cd12 service cinder] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:22.155424 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1024/2040] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:11:21 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 22:11:22.281244 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1017/2041] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:11:22 2022] PUT /identity/v3/projects/0d7cfb6fbdb94c16930bdf47ce8f8f47/users/b3873412e4fa4f5aae244082c3a196d4/roles/56e4a96094e9434cbbafb22579c034f5 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:11:22.292530 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05aa4b3-4b11-442a-b3d7-2793f7d1c166 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:22.292814 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05aa4b3-4b11-442a-b3d7-2793f7d1c166 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:22.293086 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05aa4b3-4b11-442a-b3d7-2793f7d1c166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:22.320723 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-b05aa4b3-4b11-442a-b3d7-2793f7d1c166 None None] MFA Rules not processed for user `b3873412e4fa4f5aae244082c3a196d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:11:22.321932 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-b05aa4b3-4b11-442a-b3d7-2793f7d1c166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:22.383948 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbd577b-ec6c-4a18-a85b-c11eaa50b611 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:22.394977 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1025/2042] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:11:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:11:22.405093 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05aa4b3-4b11-442a-b3d7-2793f7d1c166 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:22.448100 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbd577b-ec6c-4a18-a85b-c11eaa50b611 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:22.452662 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65bd4a3-14cc-405b-bfa0-c274c083877b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa71c71de263478da234f3c5716a6e5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:22.453219 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65bd4a3-14cc-405b-bfa0-c274c083877b service cinder] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:22.453503 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65bd4a3-14cc-405b-bfa0-c274c083877b service cinder] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:22.453780 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65bd4a3-14cc-405b-bfa0-c274c083877b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:22.455596 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-a65bd4a3-14cc-405b-bfa0-c274c083877b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:22.475304 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05aa4b3-4b11-442a-b3d7-2793f7d1c166 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:22.485052 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5127b17-c744-401d-be68-8ac094c34d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:22.485656 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5127b17-c744-401d-be68-8ac094c34d1b admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:22.485935 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5127b17-c744-401d-be68-8ac094c34d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:22.486209 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5127b17-c744-401d-be68-8ac094c34d1b admin admin] PATH_INFO: `/v3/projects` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:22.487121 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5127b17-c744-401d-be68-8ac094c34d1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:22.489021 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5127b17-c744-401d-be68-8ac094c34d1b admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:22.523801 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65bd4a3-14cc-405b-bfa0-c274c083877b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:22.525360 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65bd4a3-14cc-405b-bfa0-c274c083877b service cinder] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:22.536246 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1018/2043] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:11:22 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 22:11:22.660915 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1026/2044] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:11:22 2022] POST /identity/v3/projects => generated 371 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 22:11:22.665069 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65bd4a3-14cc-405b-bfa0-c274c083877b service cinder] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:22.677713 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65bd4a3-14cc-405b-bfa0-c274c083877b service cinder] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:22.749834 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb42e4da-58e4-4be8-bc2f-214bd24cf94c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:22.750685 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb42e4da-58e4-4be8-bc2f-214bd24cf94c admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:22.751057 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb42e4da-58e4-4be8-bc2f-214bd24cf94c admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:22.751452 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb42e4da-58e4-4be8-bc2f-214bd24cf94c admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:22.752464 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb42e4da-58e4-4be8-bc2f-214bd24cf94c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:22.754918 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb42e4da-58e4-4be8-bc2f-214bd24cf94c admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:22.792216 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5127b17-c744-401d-be68-8ac094c34d1b admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:22.794294 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:22.796812 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-a5127b17-c744-401d-be68-8ac094c34d1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:22.813959 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5127b17-c744-401d-be68-8ac094c34d1b admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:22.824970 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd155c4b-6ff6-48eb-ae9a-0f4cadaac508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:22.825804 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd155c4b-6ff6-48eb-ae9a-0f4cadaac508 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:22.826227 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd155c4b-6ff6-48eb-ae9a-0f4cadaac508 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:22.826652 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd155c4b-6ff6-48eb-ae9a-0f4cadaac508 admin admin] PATH_INFO: `/v3/users/1db1473cf322488dbb7b9f26419207fa` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:22.828188 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd155c4b-6ff6-48eb-ae9a-0f4cadaac508 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1db1473cf322488dbb7b9f26419207fa)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:22.830358 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd155c4b-6ff6-48eb-ae9a-0f4cadaac508 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:22.968023 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:22.974366 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1019/2045] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:11:22 2022] POST /identity/v3/users => generated 292 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 22:11:22.978509 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb42e4da-58e4-4be8-bc2f-214bd24cf94c admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:22.988500 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb42e4da-58e4-4be8-bc2f-214bd24cf94c admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:22.993298 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbd63e3-1aca-486f-aad0-55bf96f0a2e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:22.993804 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbd63e3-1aca-486f-aad0-55bf96f0a2e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:22.994078 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbd63e3-1aca-486f-aad0-55bf96f0a2e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:22.994348 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbd63e3-1aca-486f-aad0-55bf96f0a2e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:22.994987 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbd63e3-1aca-486f-aad0-55bf96f0a2e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:22.996822 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbd63e3-1aca-486f-aad0-55bf96f0a2e4 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:23.010522 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1020/2046] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:11:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 22:11:23.012252 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbd63e3-1aca-486f-aad0-55bf96f0a2e4 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:23.022379 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbd63e3-1aca-486f-aad0-55bf96f0a2e4 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:23.027371 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb35f9f-7e88-46bf-8a03-a3eeb0de5e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:23.029160 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb35f9f-7e88-46bf-8a03-a3eeb0de5e9d admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:23.032353 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb35f9f-7e88-46bf-8a03-a3eeb0de5e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:23.032353 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb35f9f-7e88-46bf-8a03-a3eeb0de5e9d admin admin] PATH_INFO: `/v3/projects/0574f5cee7d54d0d9b2a3fe259babda5/users/7709b37725a74594af5181bc1db8bf8c/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:23.046862 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb35f9f-7e88-46bf-8a03-a3eeb0de5e9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0574f5cee7d54d0d9b2a3fe259babda5, user_id=7709b37725a74594af5181bc1db8bf8c, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:23.049831 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb35f9f-7e88-46bf-8a03-a3eeb0de5e9d admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:23.099901 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-bd155c4b-6ff6-48eb-ae9a-0f4cadaac508 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1db1473cf322488dbb7b9f26419207fa'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:23.112335 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-bd155c4b-6ff6-48eb-ae9a-0f4cadaac508 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1db1473cf322488dbb7b9f26419207fa'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:23.120867 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1021/2047] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:11:23 2022] PUT /identity/v3/projects/0574f5cee7d54d0d9b2a3fe259babda5/users/7709b37725a74594af5181bc1db8bf8c/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:11:23.124902 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb35f9f-7e88-46bf-8a03-a3eeb0de5e9d admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:23.185313 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb35f9f-7e88-46bf-8a03-a3eeb0de5e9d admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:23.190155 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b20444b-0f20-4cb1-92f6-820406141e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:23.190757 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b20444b-0f20-4cb1-92f6-820406141e30 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:23.191034 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b20444b-0f20-4cb1-92f6-820406141e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:23.191304 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b20444b-0f20-4cb1-92f6-820406141e30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:23.192286 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b20444b-0f20-4cb1-92f6-820406141e30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:23.193737 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b20444b-0f20-4cb1-92f6-820406141e30 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:23.205064 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1022/2048] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:11:23 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 22:11:23.209548 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b20444b-0f20-4cb1-92f6-820406141e30 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:23.220896 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-bd155c4b-6ff6-48eb-ae9a-0f4cadaac508 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1db1473cf322488dbb7b9f26419207fa'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:23.227691 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b20444b-0f20-4cb1-92f6-820406141e30 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:23.233251 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6d24c4-d4ca-44c3-a9aa-e9fcdabbe1fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:23.233978 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d24c4-d4ca-44c3-a9aa-e9fcdabbe1fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:23.234258 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d24c4-d4ca-44c3-a9aa-e9fcdabbe1fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:23.234527 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d24c4-d4ca-44c3-a9aa-e9fcdabbe1fb admin admin] PATH_INFO: `/v3/projects/0574f5cee7d54d0d9b2a3fe259babda5/users/7709b37725a74594af5181bc1db8bf8c/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:23.236481 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6d24c4-d4ca-44c3-a9aa-e9fcdabbe1fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=0574f5cee7d54d0d9b2a3fe259babda5, user_id=7709b37725a74594af5181bc1db8bf8c, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:23.239371 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6d24c4-d4ca-44c3-a9aa-e9fcdabbe1fb admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:23.250589 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-bd155c4b-6ff6-48eb-ae9a-0f4cadaac508 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1db1473cf322488dbb7b9f26419207fa'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:23.260940 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1023/2049] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:11:23 2022] PUT /identity/v3/projects/0574f5cee7d54d0d9b2a3fe259babda5/users/7709b37725a74594af5181bc1db8bf8c/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 22:11:23.261163 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-bd155c4b-6ff6-48eb-ae9a-0f4cadaac508 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1db1473cf322488dbb7b9f26419207fa'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:23.261256 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6d24c4-d4ca-44c3-a9aa-e9fcdabbe1fb admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:23.261256 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-6f6d24c4-d4ca-44c3-a9aa-e9fcdabbe1fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:23.261413 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1027/2050] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:11:22 2022] DELETE /identity/v3/users/1db1473cf322488dbb7b9f26419207fa => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 22:11:23.271638 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f4efde-c44e-4e84-851d-e0e2dff31765 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:23.272267 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f4efde-c44e-4e84-851d-e0e2dff31765 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:23.272566 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f4efde-c44e-4e84-851d-e0e2dff31765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:23.308054 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-12f4efde-c44e-4e84-851d-e0e2dff31765 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:11:23.309844 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-12f4efde-c44e-4e84-851d-e0e2dff31765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:23.338303 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6d24c4-d4ca-44c3-a9aa-e9fcdabbe1fb admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:23.379713 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1028/2051] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:11:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:11:23.381138 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58a6b80-b7e7-49d8-8c49-7a5d393e58b3 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:23.383218 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58a6b80-b7e7-49d8-8c49-7a5d393e58b3 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:23.383218 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58a6b80-b7e7-49d8-8c49-7a5d393e58b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:23.421100 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-f58a6b80-b7e7-49d8-8c49-7a5d393e58b3 None None] MFA Rules not processed for user `7709b37725a74594af5181bc1db8bf8c`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:11:23.422596 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-f58a6b80-b7e7-49d8-8c49-7a5d393e58b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:23.440362 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d3d422-8abf-4b4b-8f36-61673c83313b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:23.440717 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d3d422-8abf-4b4b-8f36-61673c83313b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:23.441080 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d3d422-8abf-4b4b-8f36-61673c83313b admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:23.441359 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d3d422-8abf-4b4b-8f36-61673c83313b admin admin] PATH_INFO: `/v3/users/653fb6b57de4454da7e85479124e64b1` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:23.442528 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d3d422-8abf-4b4b-8f36-61673c83313b admin admin] RBAC: Authorizing `identity:delete_user(user_id=653fb6b57de4454da7e85479124e64b1)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:23.444538 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d3d422-8abf-4b4b-8f36-61673c83313b admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:23.466216 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:23.511128 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1029/2052] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:11:23 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:11:23.514913 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58a6b80-b7e7-49d8-8c49-7a5d393e58b3 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:23.514913 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-f58a6b80-b7e7-49d8-8c49-7a5d393e58b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:23.572089 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58a6b80-b7e7-49d8-8c49-7a5d393e58b3 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:23.577139 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3afcca-b876-42f0-9774-5d861477d25a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:23.577745 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3afcca-b876-42f0-9774-5d861477d25a service neutron] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:23.578017 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3afcca-b876-42f0-9774-5d861477d25a service neutron] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:23.578281 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3afcca-b876-42f0-9774-5d861477d25a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:23.587070 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3afcca-b876-42f0-9774-5d861477d25a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:23.588534 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3afcca-b876-42f0-9774-5d861477d25a service neutron] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:23.598857 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1030/2053] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:11:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:11:23.599678 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3afcca-b876-42f0-9774-5d861477d25a service neutron] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:23.601213 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-ea3afcca-b876-42f0-9774-5d861477d25a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:23.658411 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-d0d3d422-8abf-4b4b-8f36-61673c83313b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '653fb6b57de4454da7e85479124e64b1'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:23.658904 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3afcca-b876-42f0-9774-5d861477d25a service neutron] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:23.663158 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998686aa-f234-426e-9e0f-c651137c1fe4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa71c71de263478da234f3c5716a6e5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:23.663761 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998686aa-f234-426e-9e0f-c651137c1fe4 service cinder] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:23.664038 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998686aa-f234-426e-9e0f-c651137c1fe4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:23.664314 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998686aa-f234-426e-9e0f-c651137c1fe4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:23.666453 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-998686aa-f234-426e-9e0f-c651137c1fe4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:23.675641 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-d0d3d422-8abf-4b4b-8f36-61673c83313b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '653fb6b57de4454da7e85479124e64b1'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:23.682873 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-d0d3d422-8abf-4b4b-8f36-61673c83313b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '653fb6b57de4454da7e85479124e64b1'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:23.683845 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-d0d3d422-8abf-4b4b-8f36-61673c83313b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '653fb6b57de4454da7e85479124e64b1'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:23.684864 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-d0d3d422-8abf-4b4b-8f36-61673c83313b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '653fb6b57de4454da7e85479124e64b1'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:23.727625 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998686aa-f234-426e-9e0f-c651137c1fe4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:23.729017 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998686aa-f234-426e-9e0f-c651137c1fe4 service cinder] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:23.738239 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1031/2054] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:11:23 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:11:23.755216 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:23.760639 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1024/2055] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:11:23 2022] DELETE /identity/v3/users/653fb6b57de4454da7e85479124e64b1 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:11:23.769496 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5153d81-7aac-4122-9ef3-dd06983835f1 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:23.769945 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5153d81-7aac-4122-9ef3-dd06983835f1 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:23.770363 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5153d81-7aac-4122-9ef3-dd06983835f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:23.809316 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-e5153d81-7aac-4122-9ef3-dd06983835f1 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:11:23.826405 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-e5153d81-7aac-4122-9ef3-dd06983835f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:23.853634 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:23.921103 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1032/2056] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:11:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:11:23.931219 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d3d422-8abf-4b4b-8f36-61673c83313b admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:23.932804 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-d0d3d422-8abf-4b4b-8f36-61673c83313b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:24.001751 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d3d422-8abf-4b4b-8f36-61673c83313b admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:24.006462 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcbcc57-2ace-4411-8a38-6fd57caac6fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:24.007109 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcbcc57-2ace-4411-8a38-6fd57caac6fe service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:24.007684 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcbcc57-2ace-4411-8a38-6fd57caac6fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:24.007684 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcbcc57-2ace-4411-8a38-6fd57caac6fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:24.019283 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcbcc57-2ace-4411-8a38-6fd57caac6fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:24.020871 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcbcc57-2ace-4411-8a38-6fd57caac6fe service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:24.040566 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1025/2057] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:11:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:11:24.126659 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aeb6a8-9bba-4b87-8194-ce71ecdd4294 None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:24.126954 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aeb6a8-9bba-4b87-8194-ce71ecdd4294 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:24.127235 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aeb6a8-9bba-4b87-8194-ce71ecdd4294 None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:24.128648 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1033/2058] 10.222.0.18 () {54 vars in 921 bytes} [Sat Aug 6 22:11:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:11:24.274750 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a356ae-2b67-4238-9655-9cc6ab77175d None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:24.274750 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a356ae-2b67-4238-9655-9cc6ab77175d None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:24.274750 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a356ae-2b67-4238-9655-9cc6ab77175d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:24.274750 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-b4a356ae-2b67-4238-9655-9cc6ab77175d None None] MFA Rules not processed for user `2a1319d33e9144859c6010843be312a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:11:24.274750 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-b4a356ae-2b67-4238-9655-9cc6ab77175d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:24.274750 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1026/2059] 10.222.0.18 () {58 vars in 1018 bytes} [Sat Aug 6 22:11:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 22:11:24.275978 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aeb6a8-9bba-4b87-8194-ce71ecdd4294 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:24.275978 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-61aeb6a8-9bba-4b87-8194-ce71ecdd4294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:24.275978 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aeb6a8-9bba-4b87-8194-ce71ecdd4294 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:24.276232 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eb2d65-b316-427e-859c-5f95680cceaa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a1319d33e9144859c6010843be312a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:24.276232 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eb2d65-b316-427e-859c-5f95680cceaa service nova] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:24.276232 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eb2d65-b316-427e-859c-5f95680cceaa service nova] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:24.276232 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eb2d65-b316-427e-859c-5f95680cceaa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:24.276593 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eb2d65-b316-427e-859c-5f95680cceaa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:24.276593 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eb2d65-b316-427e-859c-5f95680cceaa service nova] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:24.287755 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1034/2060] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:11:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:11:24.374670 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a356ae-2b67-4238-9655-9cc6ab77175d None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:24.377101 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-b4a356ae-2b67-4238-9655-9cc6ab77175d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:24.584346 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a356ae-2b67-4238-9655-9cc6ab77175d None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:24.584346 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adcf2c0-6112-4c1c-be15-3123843b5ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:24.584346 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adcf2c0-6112-4c1c-be15-3123843b5ae3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:24.584346 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adcf2c0-6112-4c1c-be15-3123843b5ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:24.587504 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adcf2c0-6112-4c1c-be15-3123843b5ae3 admin admin] PATH_INFO: `/v3/projects/e49c8f3f6e10431282d64c4036246499` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:24.587504 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adcf2c0-6112-4c1c-be15-3123843b5ae3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e49c8f3f6e10431282d64c4036246499)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:24.587504 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adcf2c0-6112-4c1c-be15-3123843b5ae3 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:25.544589 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:25.626941 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-2adcf2c0-6112-4c1c-be15-3123843b5ae3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e49c8f3f6e10431282d64c4036246499', 'request_id': 'req-2adcf2c0-6112-4c1c-be15-3123843b5ae3'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:25.795872 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-2adcf2c0-6112-4c1c-be15-3123843b5ae3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e49c8f3f6e10431282d64c4036246499', 'request_id': 'req-2adcf2c0-6112-4c1c-be15-3123843b5ae3'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:25.806534 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-2adcf2c0-6112-4c1c-be15-3123843b5ae3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e49c8f3f6e10431282d64c4036246499', 'request_id': 'req-2adcf2c0-6112-4c1c-be15-3123843b5ae3'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:25.820583 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-2adcf2c0-6112-4c1c-be15-3123843b5ae3 admin admin] The token cache is being invalidate because project e49c8f3f6e10431282d64c4036246499 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:11:25.820818 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-2adcf2c0-6112-4c1c-be15-3123843b5ae3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:25.822007 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-2adcf2c0-6112-4c1c-be15-3123843b5ae3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:25.824032 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1027/2061] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:11:24 2022] DELETE /identity/v3/projects/e49c8f3f6e10431282d64c4036246499 => generated 0 bytes in 1450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:11:25.829382 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eb2d65-b316-427e-859c-5f95680cceaa service nova] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:25.842724 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eb2d65-b316-427e-859c-5f95680cceaa service nova] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:25.878096 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adcf2c0-6112-4c1c-be15-3123843b5ae3 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:25.887331 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-2adcf2c0-6112-4c1c-be15-3123843b5ae3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:25.986299 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e1c5fd-cc43-45b3-ba10-c45f3752270e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:25.987203 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e1c5fd-cc43-45b3-ba10-c45f3752270e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:25.987629 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e1c5fd-cc43-45b3-ba10-c45f3752270e admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:25.988066 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e1c5fd-cc43-45b3-ba10-c45f3752270e admin admin] PATH_INFO: `/v3/users/493c82a5168a44a595d2bb5cac94dbbb` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:25.988300 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adcf2c0-6112-4c1c-be15-3123843b5ae3 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:25.989938 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e1c5fd-cc43-45b3-ba10-c45f3752270e admin admin] RBAC: Authorizing `identity:delete_user(user_id=493c82a5168a44a595d2bb5cac94dbbb)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:25.992379 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e1c5fd-cc43-45b3-ba10-c45f3752270e admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:25.994245 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a6eb07-4b1f-4a8e-b47c-286f5fd25438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:25.995655 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a6eb07-4b1f-4a8e-b47c-286f5fd25438 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:25.996109 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a6eb07-4b1f-4a8e-b47c-286f5fd25438 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:25.996608 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a6eb07-4b1f-4a8e-b47c-286f5fd25438 admin admin] PATH_INFO: `/v3/projects/9ea3017518bb4947b33d7ee8d8ffd952` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:25.998384 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a6eb07-4b1f-4a8e-b47c-286f5fd25438 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ea3017518bb4947b33d7ee8d8ffd952)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:26.002203 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a6eb07-4b1f-4a8e-b47c-286f5fd25438 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:26.279466 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-11e1c5fd-cc43-45b3-ba10-c45f3752270e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '493c82a5168a44a595d2bb5cac94dbbb'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:26.281048 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-c6a6eb07-4b1f-4a8e-b47c-286f5fd25438 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ea3017518bb4947b33d7ee8d8ffd952', 'request_id': 'req-c6a6eb07-4b1f-4a8e-b47c-286f5fd25438'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:26.292712 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-11e1c5fd-cc43-45b3-ba10-c45f3752270e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '493c82a5168a44a595d2bb5cac94dbbb'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:26.400794 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-c6a6eb07-4b1f-4a8e-b47c-286f5fd25438 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ea3017518bb4947b33d7ee8d8ffd952', 'request_id': 'req-c6a6eb07-4b1f-4a8e-b47c-286f5fd25438'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:26.411639 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-c6a6eb07-4b1f-4a8e-b47c-286f5fd25438 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ea3017518bb4947b33d7ee8d8ffd952', 'request_id': 'req-c6a6eb07-4b1f-4a8e-b47c-286f5fd25438'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:26.425078 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-c6a6eb07-4b1f-4a8e-b47c-286f5fd25438 admin admin] The token cache is being invalidate because project 9ea3017518bb4947b33d7ee8d8ffd952 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:11:26.425438 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-c6a6eb07-4b1f-4a8e-b47c-286f5fd25438 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:26.431196 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-c6a6eb07-4b1f-4a8e-b47c-286f5fd25438 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:26.432790 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1028/2062] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:11:25 2022] DELETE /identity/v3/projects/9ea3017518bb4947b33d7ee8d8ffd952 => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 22:11:26.460014 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a6eb07-4b1f-4a8e-b47c-286f5fd25438 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:26.461403 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-c6a6eb07-4b1f-4a8e-b47c-286f5fd25438 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:26.529481 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a6eb07-4b1f-4a8e-b47c-286f5fd25438 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:26.535232 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc43e27-3697-43e5-a061-8dcf4ed8edf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:26.535860 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc43e27-3697-43e5-a061-8dcf4ed8edf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:26.536136 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc43e27-3697-43e5-a061-8dcf4ed8edf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:26.536476 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc43e27-3697-43e5-a061-8dcf4ed8edf1 admin admin] PATH_INFO: `/v3/users/e31371fe75084e6596c66995bde3cce7` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:26.537619 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc43e27-3697-43e5-a061-8dcf4ed8edf1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e31371fe75084e6596c66995bde3cce7)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:26.539548 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc43e27-3697-43e5-a061-8dcf4ed8edf1 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:26.560432 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-11e1c5fd-cc43-45b3-ba10-c45f3752270e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '493c82a5168a44a595d2bb5cac94dbbb'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:26.569009 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-11e1c5fd-cc43-45b3-ba10-c45f3752270e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '493c82a5168a44a595d2bb5cac94dbbb'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:26.569731 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-11e1c5fd-cc43-45b3-ba10-c45f3752270e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '493c82a5168a44a595d2bb5cac94dbbb'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:26.575984 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1035/2063] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:11:25 2022] DELETE /identity/v3/users/493c82a5168a44a595d2bb5cac94dbbb => generated 0 bytes in 747 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:11:26.749607 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:26.840922 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-3bc43e27-3697-43e5-a061-8dcf4ed8edf1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e31371fe75084e6596c66995bde3cce7'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:26.853629 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-3bc43e27-3697-43e5-a061-8dcf4ed8edf1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e31371fe75084e6596c66995bde3cce7'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:26.861280 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-3bc43e27-3697-43e5-a061-8dcf4ed8edf1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e31371fe75084e6596c66995bde3cce7'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:26.862161 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-3bc43e27-3697-43e5-a061-8dcf4ed8edf1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e31371fe75084e6596c66995bde3cce7'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:26.862873 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-3bc43e27-3697-43e5-a061-8dcf4ed8edf1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e31371fe75084e6596c66995bde3cce7'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:26.961085 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1029/2064] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:11:26 2022] DELETE /identity/v3/users/e31371fe75084e6596c66995bde3cce7 => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 22:11:27.109776 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:27.332181 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e1c5fd-cc43-45b3-ba10-c45f3752270e admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:27.333610 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-11e1c5fd-cc43-45b3-ba10-c45f3752270e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:27.463818 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e1c5fd-cc43-45b3-ba10-c45f3752270e admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:27.526921 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ff8e49-9125-453e-be41-06b42b1b50ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:27.527874 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ff8e49-9125-453e-be41-06b42b1b50ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:27.528215 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ff8e49-9125-453e-be41-06b42b1b50ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:27.528623 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ff8e49-9125-453e-be41-06b42b1b50ca admin admin] PATH_INFO: `/v3/projects/bbff8826d2d74874a46fdef774001fa8` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:27.529798 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ff8e49-9125-453e-be41-06b42b1b50ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbff8826d2d74874a46fdef774001fa8)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:27.531864 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ff8e49-9125-453e-be41-06b42b1b50ca admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:27.674686 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-a3ff8e49-9125-453e-be41-06b42b1b50ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbff8826d2d74874a46fdef774001fa8', 'request_id': 'req-a3ff8e49-9125-453e-be41-06b42b1b50ca'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:27.705854 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc43e27-3697-43e5-a061-8dcf4ed8edf1 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:27.707516 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-3bc43e27-3697-43e5-a061-8dcf4ed8edf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:27.722221 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc43e27-3697-43e5-a061-8dcf4ed8edf1 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:27.779660 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293e796d-da18-487f-8dba-bf4c953d51df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:27.780316 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293e796d-da18-487f-8dba-bf4c953d51df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:27.780586 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293e796d-da18-487f-8dba-bf4c953d51df admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:27.780863 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293e796d-da18-487f-8dba-bf4c953d51df admin admin] PATH_INFO: `/v3/projects/4a4e40c23f8544e3a65bb78c085822a7` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:27.782294 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293e796d-da18-487f-8dba-bf4c953d51df admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a4e40c23f8544e3a65bb78c085822a7)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:27.783237 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-a3ff8e49-9125-453e-be41-06b42b1b50ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbff8826d2d74874a46fdef774001fa8', 'request_id': 'req-a3ff8e49-9125-453e-be41-06b42b1b50ca'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:27.784217 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293e796d-da18-487f-8dba-bf4c953d51df admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:27.790681 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-a3ff8e49-9125-453e-be41-06b42b1b50ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbff8826d2d74874a46fdef774001fa8', 'request_id': 'req-a3ff8e49-9125-453e-be41-06b42b1b50ca'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:27.802292 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-a3ff8e49-9125-453e-be41-06b42b1b50ca admin admin] The token cache is being invalidate because project bbff8826d2d74874a46fdef774001fa8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:11:27.802762 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-a3ff8e49-9125-453e-be41-06b42b1b50ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:27.803421 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-a3ff8e49-9125-453e-be41-06b42b1b50ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:27.804986 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1036/2065] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:11:27 2022] DELETE /identity/v3/projects/bbff8826d2d74874a46fdef774001fa8 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:11:27.809621 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ff8e49-9125-453e-be41-06b42b1b50ca admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:27.810948 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-a3ff8e49-9125-453e-be41-06b42b1b50ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:27.823021 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ff8e49-9125-453e-be41-06b42b1b50ca admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:27.828185 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e68d56b-49c5-4ccc-ba9f-21bb42e10a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:27.828841 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e68d56b-49c5-4ccc-ba9f-21bb42e10a0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:27.829121 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e68d56b-49c5-4ccc-ba9f-21bb42e10a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:27.829395 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e68d56b-49c5-4ccc-ba9f-21bb42e10a0c admin admin] PATH_INFO: `/v3/users/2c3a6bf7a11a41c19a027738b6fe32e9` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:27.830523 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e68d56b-49c5-4ccc-ba9f-21bb42e10a0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c3a6bf7a11a41c19a027738b6fe32e9)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:27.832443 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e68d56b-49c5-4ccc-ba9f-21bb42e10a0c admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:27.897419 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-293e796d-da18-487f-8dba-bf4c953d51df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a4e40c23f8544e3a65bb78c085822a7', 'request_id': 'req-293e796d-da18-487f-8dba-bf4c953d51df'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:27.989038 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-293e796d-da18-487f-8dba-bf4c953d51df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a4e40c23f8544e3a65bb78c085822a7', 'request_id': 'req-293e796d-da18-487f-8dba-bf4c953d51df'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:27.995106 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-293e796d-da18-487f-8dba-bf4c953d51df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a4e40c23f8544e3a65bb78c085822a7', 'request_id': 'req-293e796d-da18-487f-8dba-bf4c953d51df'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:28.006969 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-293e796d-da18-487f-8dba-bf4c953d51df admin admin] The token cache is being invalidate because project 4a4e40c23f8544e3a65bb78c085822a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:11:28.007250 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-293e796d-da18-487f-8dba-bf4c953d51df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:28.007864 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-293e796d-da18-487f-8dba-bf4c953d51df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:28.009378 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1030/2066] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:11:27 2022] DELETE /identity/v3/projects/4a4e40c23f8544e3a65bb78c085822a7 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 22:11:28.016087 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c690df9-80bf-4583-8a0c-95d4b9b8d140 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:28.016657 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c690df9-80bf-4583-8a0c-95d4b9b8d140 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:28.016937 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c690df9-80bf-4583-8a0c-95d4b9b8d140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:28.031457 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-3e68d56b-49c5-4ccc-ba9f-21bb42e10a0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c3a6bf7a11a41c19a027738b6fe32e9'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:28.035802 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-0c690df9-80bf-4583-8a0c-95d4b9b8d140 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:11:28.037022 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-0c690df9-80bf-4583-8a0c-95d4b9b8d140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:28.043638 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-3e68d56b-49c5-4ccc-ba9f-21bb42e10a0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c3a6bf7a11a41c19a027738b6fe32e9'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:28.097358 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1031/2067] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:11:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 22:11:28.106999 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb94e4d5-c629-45e9-ae55-56bfe87fd192 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:28.107280 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb94e4d5-c629-45e9-ae55-56bfe87fd192 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:28.107556 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb94e4d5-c629-45e9-ae55-56bfe87fd192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:28.127435 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-cb94e4d5-c629-45e9-ae55-56bfe87fd192 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:11:28.128846 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-cb94e4d5-c629-45e9-ae55-56bfe87fd192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:28.137874 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1032/2068] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:11:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 22:11:28.142470 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb94e4d5-c629-45e9-ae55-56bfe87fd192 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:28.148593 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-3e68d56b-49c5-4ccc-ba9f-21bb42e10a0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c3a6bf7a11a41c19a027738b6fe32e9'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:28.151714 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb94e4d5-c629-45e9-ae55-56bfe87fd192 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:28.159122 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d163e545-6b81-4141-9b77-689751cca99f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:28.159122 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d163e545-6b81-4141-9b77-689751cca99f admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:28.159122 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d163e545-6b81-4141-9b77-689751cca99f admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:28.159696 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-3e68d56b-49c5-4ccc-ba9f-21bb42e10a0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c3a6bf7a11a41c19a027738b6fe32e9'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:28.159790 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d163e545-6b81-4141-9b77-689751cca99f admin admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:28.159899 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-3e68d56b-49c5-4ccc-ba9f-21bb42e10a0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c3a6bf7a11a41c19a027738b6fe32e9'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:28.159985 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d163e545-6b81-4141-9b77-689751cca99f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 22:11:28.159985 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d163e545-6b81-4141-9b77-689751cca99f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:28.160429 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d163e545-6b81-4141-9b77-689751cca99f admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:28.165928 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1037/2069] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:11:27 2022] DELETE /identity/v3/users/2c3a6bf7a11a41c19a027738b6fe32e9 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 22:11:28.176233 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1033/2070] 10.222.0.18 () {58 vars in 1153 bytes} [Sat Aug 6 22:11:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 22:11:28.179518 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e68d56b-49c5-4ccc-ba9f-21bb42e10a0c admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:28.181063 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-3e68d56b-49c5-4ccc-ba9f-21bb42e10a0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:28.236098 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d163e545-6b81-4141-9b77-689751cca99f admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:28.240895 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-d163e545-6b81-4141-9b77-689751cca99f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:28.256346 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d163e545-6b81-4141-9b77-689751cca99f admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:28.262809 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5dc3f0-18f0-40aa-b9cb-a6d367a2dc45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:28.263980 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5dc3f0-18f0-40aa-b9cb-a6d367a2dc45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:28.264561 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5dc3f0-18f0-40aa-b9cb-a6d367a2dc45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:28.264928 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5dc3f0-18f0-40aa-b9cb-a6d367a2dc45 admin admin] PATH_INFO: `/v3/users/e9db5f5d5a774b8eb5e121bc46fb466f` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:28.266242 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5dc3f0-18f0-40aa-b9cb-a6d367a2dc45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9db5f5d5a774b8eb5e121bc46fb466f)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:28.268684 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5dc3f0-18f0-40aa-b9cb-a6d367a2dc45 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:28.270676 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e68d56b-49c5-4ccc-ba9f-21bb42e10a0c admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:28.277616 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270ca21f-3465-4bfb-b9a5-c0ee9d567f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:28.278519 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270ca21f-3465-4bfb-b9a5-c0ee9d567f58 admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:28.279069 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270ca21f-3465-4bfb-b9a5-c0ee9d567f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:28.279419 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270ca21f-3465-4bfb-b9a5-c0ee9d567f58 admin admin] PATH_INFO: `/v3/projects` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:28.280188 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270ca21f-3465-4bfb-b9a5-c0ee9d567f58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:28.283883 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270ca21f-3465-4bfb-b9a5-c0ee9d567f58 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:28.409131 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1038/2071] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:11:28 2022] POST /identity/v3/projects => generated 385 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:11:28.413765 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270ca21f-3465-4bfb-b9a5-c0ee9d567f58 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:28.480617 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270ca21f-3465-4bfb-b9a5-c0ee9d567f58 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:28.486077 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0351941-b442-45b6-8055-59cb1168606b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:28.487043 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0351941-b442-45b6-8055-59cb1168606b admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:28.487464 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0351941-b442-45b6-8055-59cb1168606b admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:28.487900 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0351941-b442-45b6-8055-59cb1168606b admin admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:28.488721 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0351941-b442-45b6-8055-59cb1168606b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:28.490772 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0351941-b442-45b6-8055-59cb1168606b admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:28.540350 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-bc5dc3f0-18f0-40aa-b9cb-a6d367a2dc45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9db5f5d5a774b8eb5e121bc46fb466f'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:28.553527 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-bc5dc3f0-18f0-40aa-b9cb-a6d367a2dc45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9db5f5d5a774b8eb5e121bc46fb466f'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:28.561843 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-bc5dc3f0-18f0-40aa-b9cb-a6d367a2dc45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9db5f5d5a774b8eb5e121bc46fb466f'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:28.562586 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-bc5dc3f0-18f0-40aa-b9cb-a6d367a2dc45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9db5f5d5a774b8eb5e121bc46fb466f'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:28.563253 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-bc5dc3f0-18f0-40aa-b9cb-a6d367a2dc45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9db5f5d5a774b8eb5e121bc46fb466f'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:28.607798 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1039/2072] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:11:28 2022] POST /identity/v3/users => generated 293 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:11:28.613857 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0351941-b442-45b6-8055-59cb1168606b admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:28.613857 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-f0351941-b442-45b6-8055-59cb1168606b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:28.626327 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0351941-b442-45b6-8055-59cb1168606b admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:28.631060 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee26f1d-5c97-4cd7-80c0-7331ac6e32e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:28.631712 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee26f1d-5c97-4cd7-80c0-7331ac6e32e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:28.631996 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee26f1d-5c97-4cd7-80c0-7331ac6e32e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:28.632270 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee26f1d-5c97-4cd7-80c0-7331ac6e32e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:28.632973 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee26f1d-5c97-4cd7-80c0-7331ac6e32e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:28.634643 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee26f1d-5c97-4cd7-80c0-7331ac6e32e6 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:28.649291 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1034/2073] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:11:28 2022] DELETE /identity/v3/users/e9db5f5d5a774b8eb5e121bc46fb466f => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:11:28.651133 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1040/2074] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:11:28 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 22:11:28.654921 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17812e79-e815-4e9c-89d1-1c9d63478442 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:28.655630 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17812e79-e815-4e9c-89d1-1c9d63478442 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:28.655853 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee26f1d-5c97-4cd7-80c0-7331ac6e32e6 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:28.656647 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17812e79-e815-4e9c-89d1-1c9d63478442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:28.666088 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee26f1d-5c97-4cd7-80c0-7331ac6e32e6 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:28.688186 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-17812e79-e815-4e9c-89d1-1c9d63478442 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:11:28.689610 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-17812e79-e815-4e9c-89d1-1c9d63478442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:28.728044 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:28.742634 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e556ff-8f3b-408a-9bdd-690f9bcb2182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:28.743330 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e556ff-8f3b-408a-9bdd-690f9bcb2182 admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:28.743567 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e556ff-8f3b-408a-9bdd-690f9bcb2182 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:28.743838 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e556ff-8f3b-408a-9bdd-690f9bcb2182 admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:28.744725 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e556ff-8f3b-408a-9bdd-690f9bcb2182 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:28.746704 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e556ff-8f3b-408a-9bdd-690f9bcb2182 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:28.758919 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1041/2075] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:11:28 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:11:28.761064 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e556ff-8f3b-408a-9bdd-690f9bcb2182 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:28.763016 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-a7e556ff-8f3b-408a-9bdd-690f9bcb2182 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:28.775858 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e556ff-8f3b-408a-9bdd-690f9bcb2182 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:28.781368 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df690d2c-e172-4b98-8a1e-5fbe6538e404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:28.782004 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df690d2c-e172-4b98-8a1e-5fbe6538e404 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:28.782658 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df690d2c-e172-4b98-8a1e-5fbe6538e404 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:28.782658 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df690d2c-e172-4b98-8a1e-5fbe6538e404 admin admin] PATH_INFO: `/v3/projects/9f3e1902761e4ffd962c750c74740891` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:28.783763 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df690d2c-e172-4b98-8a1e-5fbe6538e404 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f3e1902761e4ffd962c750c74740891)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:28.784714 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1035/2076] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:11:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:11:28.785899 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df690d2c-e172-4b98-8a1e-5fbe6538e404 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:28.786491 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17812e79-e815-4e9c-89d1-1c9d63478442 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:28.797064 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17812e79-e815-4e9c-89d1-1c9d63478442 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:28.801891 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f488849f-70f9-48c7-a222-9fdd19ed61f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:28.802551 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f488849f-70f9-48c7-a222-9fdd19ed61f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:28.802830 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f488849f-70f9-48c7-a222-9fdd19ed61f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:28.803101 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f488849f-70f9-48c7-a222-9fdd19ed61f5 admin admin] PATH_INFO: `/v3/projects/72806f3bf23a42c58952a0f3b6312ba6/users/4b065a4aa3424317a646faa80af11016/roles/56e4a96094e9434cbbafb22579c034f5` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:28.813830 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:28.820716 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f488849f-70f9-48c7-a222-9fdd19ed61f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72806f3bf23a42c58952a0f3b6312ba6, user_id=4b065a4aa3424317a646faa80af11016, role_id=56e4a96094e9434cbbafb22579c034f5)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:28.823423 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f488849f-70f9-48c7-a222-9fdd19ed61f5 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:28.895443 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1036/2077] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:11:28 2022] PUT /identity/v3/projects/72806f3bf23a42c58952a0f3b6312ba6/users/4b065a4aa3424317a646faa80af11016/roles/56e4a96094e9434cbbafb22579c034f5 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:11:28.896489 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f488849f-70f9-48c7-a222-9fdd19ed61f5 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:28.901872 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-f488849f-70f9-48c7-a222-9fdd19ed61f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:28.906697 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-df690d2c-e172-4b98-8a1e-5fbe6538e404 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f3e1902761e4ffd962c750c74740891', 'request_id': 'req-df690d2c-e172-4b98-8a1e-5fbe6538e404'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:28.963381 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f488849f-70f9-48c7-a222-9fdd19ed61f5 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:28.983293 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-df690d2c-e172-4b98-8a1e-5fbe6538e404 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f3e1902761e4ffd962c750c74740891', 'request_id': 'req-df690d2c-e172-4b98-8a1e-5fbe6538e404'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:28.990137 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-df690d2c-e172-4b98-8a1e-5fbe6538e404 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f3e1902761e4ffd962c750c74740891', 'request_id': 'req-df690d2c-e172-4b98-8a1e-5fbe6538e404'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:29.003135 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-df690d2c-e172-4b98-8a1e-5fbe6538e404 admin admin] The token cache is being invalidate because project 9f3e1902761e4ffd962c750c74740891 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:11:29.003421 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-df690d2c-e172-4b98-8a1e-5fbe6538e404 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:29.004087 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-df690d2c-e172-4b98-8a1e-5fbe6538e404 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:29.005643 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1042/2078] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:11:28 2022] DELETE /identity/v3/projects/9f3e1902761e4ffd962c750c74740891 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:11:29.011366 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b1fd0e-31b9-4e38-8f72-d428e4e62aed None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:29.011691 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b1fd0e-31b9-4e38-8f72-d428e4e62aed None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:29.012215 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b1fd0e-31b9-4e38-8f72-d428e4e62aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:29.014324 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a533e2fc-3618-4b81-bb31-25f00418bf73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:29.017079 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a533e2fc-3618-4b81-bb31-25f00418bf73 service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:29.017079 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a533e2fc-3618-4b81-bb31-25f00418bf73 service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:29.017079 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a533e2fc-3618-4b81-bb31-25f00418bf73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:29.017347 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-a533e2fc-3618-4b81-bb31-25f00418bf73 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:29.039804 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-46b1fd0e-31b9-4e38-8f72-d428e4e62aed None None] MFA Rules not processed for user `4b065a4aa3424317a646faa80af11016`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:11:29.042292 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-46b1fd0e-31b9-4e38-8f72-d428e4e62aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:29.082182 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a533e2fc-3618-4b81-bb31-25f00418bf73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:29.083627 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a533e2fc-3618-4b81-bb31-25f00418bf73 service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:29.093510 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1037/2079] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:11:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:11:29.094729 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43990681-1657-4d97-b9d7-69edc9b4d043 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:29.095016 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43990681-1657-4d97-b9d7-69edc9b4d043 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:29.095289 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43990681-1657-4d97-b9d7-69edc9b4d043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:29.108447 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1043/2080] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:11:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:11:29.118824 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-43990681-1657-4d97-b9d7-69edc9b4d043 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:11:29.119568 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b1fd0e-31b9-4e38-8f72-d428e4e62aed None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:29.120395 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-43990681-1657-4d97-b9d7-69edc9b4d043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:29.121547 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-46b1fd0e-31b9-4e38-8f72-d428e4e62aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:29.131527 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1038/2081] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:11:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:11:29.140881 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc38c705-15f3-48f2-b6ef-12020e7d083e None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:29.141187 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc38c705-15f3-48f2-b6ef-12020e7d083e None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:29.142202 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc38c705-15f3-48f2-b6ef-12020e7d083e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:29.163429 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-dc38c705-15f3-48f2-b6ef-12020e7d083e None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:11:29.164760 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-dc38c705-15f3-48f2-b6ef-12020e7d083e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:29.176965 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1039/2082] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:11:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:11:29.178358 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc38c705-15f3-48f2-b6ef-12020e7d083e None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:29.188392 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc38c705-15f3-48f2-b6ef-12020e7d083e None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:29.190505 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b1fd0e-31b9-4e38-8f72-d428e4e62aed None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:29.193727 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1117340-af4a-4c90-9330-4cddb1adba0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:29.194355 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1117340-af4a-4c90-9330-4cddb1adba0d admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:29.194628 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1117340-af4a-4c90-9330-4cddb1adba0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:29.194746 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b343c854-90c7-4428-9257-cd582123f5e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa71c71de263478da234f3c5716a6e5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:29.195087 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1117340-af4a-4c90-9330-4cddb1adba0d admin admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:29.195423 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b343c854-90c7-4428-9257-cd582123f5e9 service cinder] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:29.195580 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1117340-af4a-4c90-9330-4cddb1adba0d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 22:11:29.195688 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b343c854-90c7-4428-9257-cd582123f5e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:29.195880 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1117340-af4a-4c90-9330-4cddb1adba0d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:29.195973 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b343c854-90c7-4428-9257-cd582123f5e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:29.197505 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1117340-af4a-4c90-9330-4cddb1adba0d admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:29.205823 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b343c854-90c7-4428-9257-cd582123f5e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:29.207335 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b343c854-90c7-4428-9257-cd582123f5e9 service cinder] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:29.209466 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1040/2083] 10.222.0.18 () {58 vars in 1153 bytes} [Sat Aug 6 22:11:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 22:11:29.214131 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1117340-af4a-4c90-9330-4cddb1adba0d admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:29.217606 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1044/2084] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:11:29 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 22:11:29.228289 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1117340-af4a-4c90-9330-4cddb1adba0d admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:29.230446 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21a2da7-36ac-4fae-b2ec-a090e7cd480c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:29.231051 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21a2da7-36ac-4fae-b2ec-a090e7cd480c admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:29.231322 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21a2da7-36ac-4fae-b2ec-a090e7cd480c admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:29.231594 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21a2da7-36ac-4fae-b2ec-a090e7cd480c admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:29.232382 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21a2da7-36ac-4fae-b2ec-a090e7cd480c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:29.234335 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21a2da7-36ac-4fae-b2ec-a090e7cd480c admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:29.296959 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:29.332242 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1041/2085] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:11:29 2022] POST /identity/v3/projects => generated 383 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 22:11:29.336105 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b343c854-90c7-4428-9257-cd582123f5e9 service cinder] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:29.351857 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:29.354416 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b343c854-90c7-4428-9257-cd582123f5e9 service cinder] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:29.418603 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5202fb23-64df-4fc7-97a0-d84fa21c7758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:29.419206 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5202fb23-64df-4fc7-97a0-d84fa21c7758 admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:29.419511 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5202fb23-64df-4fc7-97a0-d84fa21c7758 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:29.419751 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5202fb23-64df-4fc7-97a0-d84fa21c7758 admin admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:29.420380 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5202fb23-64df-4fc7-97a0-d84fa21c7758 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:29.422455 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5202fb23-64df-4fc7-97a0-d84fa21c7758 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:29.498618 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1045/2086] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:11:29 2022] POST /identity/v3/users => generated 292 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:11:29.502959 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21a2da7-36ac-4fae-b2ec-a090e7cd480c admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:29.514258 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21a2da7-36ac-4fae-b2ec-a090e7cd480c admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:29.519340 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a283ad-8a89-4c52-a170-2906f687af21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:29.526434 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a283ad-8a89-4c52-a170-2906f687af21 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:29.526434 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a283ad-8a89-4c52-a170-2906f687af21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:29.526434 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a283ad-8a89-4c52-a170-2906f687af21 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:29.526434 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a283ad-8a89-4c52-a170-2906f687af21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:29.526434 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a283ad-8a89-4c52-a170-2906f687af21 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:29.537189 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5202fb23-64df-4fc7-97a0-d84fa21c7758 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:29.539404 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-5202fb23-64df-4fc7-97a0-d84fa21c7758 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:29.539626 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1042/2087] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:11:29 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:11:29.543077 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a283ad-8a89-4c52-a170-2906f687af21 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:29.552976 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5202fb23-64df-4fc7-97a0-d84fa21c7758 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:29.557280 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a283ad-8a89-4c52-a170-2906f687af21 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:29.558083 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e97af1-bf94-4f0b-b90e-9219c9f5e3cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:29.559635 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e97af1-bf94-4f0b-b90e-9219c9f5e3cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:29.559981 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e97af1-bf94-4f0b-b90e-9219c9f5e3cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:29.560537 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e97af1-bf94-4f0b-b90e-9219c9f5e3cc admin admin] PATH_INFO: `/v3/projects/82c8133fb930408caf9fa6f10d489cc4` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:29.561816 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e97af1-bf94-4f0b-b90e-9219c9f5e3cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=82c8133fb930408caf9fa6f10d489cc4)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:29.564071 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e97af1-bf94-4f0b-b90e-9219c9f5e3cc admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:29.564381 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd7d3cf-1e66-4d96-bf99-f584cca0b9c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:29.565080 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd7d3cf-1e66-4d96-bf99-f584cca0b9c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:29.565373 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd7d3cf-1e66-4d96-bf99-f584cca0b9c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:29.565594 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd7d3cf-1e66-4d96-bf99-f584cca0b9c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:29.566238 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd7d3cf-1e66-4d96-bf99-f584cca0b9c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:29.567940 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd7d3cf-1e66-4d96-bf99-f584cca0b9c7 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:29.578883 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1043/2088] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:11:29 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 22:11:29.583053 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd7d3cf-1e66-4d96-bf99-f584cca0b9c7 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:29.593245 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd7d3cf-1e66-4d96-bf99-f584cca0b9c7 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:29.597933 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e756726f-9024-40f4-820c-6df75729fd97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:29.598590 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e756726f-9024-40f4-820c-6df75729fd97 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:29.598868 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e756726f-9024-40f4-820c-6df75729fd97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:29.599141 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e756726f-9024-40f4-820c-6df75729fd97 admin admin] PATH_INFO: `/v3/projects/7fc9aab747c1420da7be0cd906dc4a7d/users/963687890bee4c6d95c633361caff409/roles/56e4a96094e9434cbbafb22579c034f5` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:29.616342 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e756726f-9024-40f4-820c-6df75729fd97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fc9aab747c1420da7be0cd906dc4a7d, user_id=963687890bee4c6d95c633361caff409, role_id=56e4a96094e9434cbbafb22579c034f5)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:29.619097 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e756726f-9024-40f4-820c-6df75729fd97 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:29.711578 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:29.718025 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1044/2089] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:11:29 2022] PUT /identity/v3/projects/7fc9aab747c1420da7be0cd906dc4a7d/users/963687890bee4c6d95c633361caff409/roles/56e4a96094e9434cbbafb22579c034f5 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:11:29.724567 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-21e97af1-bf94-4f0b-b90e-9219c9f5e3cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82c8133fb930408caf9fa6f10d489cc4', 'request_id': 'req-21e97af1-bf94-4f0b-b90e-9219c9f5e3cc'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:29.729348 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a670c941-44d5-459e-b082-e78d8ddf193e None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:29.729641 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a670c941-44d5-459e-b082-e78d8ddf193e None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:29.730015 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a670c941-44d5-459e-b082-e78d8ddf193e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:29.758781 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-a670c941-44d5-459e-b082-e78d8ddf193e None None] MFA Rules not processed for user `963687890bee4c6d95c633361caff409`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:11:29.759929 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-a670c941-44d5-459e-b082-e78d8ddf193e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:29.821550 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:29.821550 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-21e97af1-bf94-4f0b-b90e-9219c9f5e3cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82c8133fb930408caf9fa6f10d489cc4', 'request_id': 'req-21e97af1-bf94-4f0b-b90e-9219c9f5e3cc'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:29.833847 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1045/2090] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:11:29 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:11:29.836097 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-21e97af1-bf94-4f0b-b90e-9219c9f5e3cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82c8133fb930408caf9fa6f10d489cc4', 'request_id': 'req-21e97af1-bf94-4f0b-b90e-9219c9f5e3cc'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:29.842243 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a670c941-44d5-459e-b082-e78d8ddf193e None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:29.845570 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-a670c941-44d5-459e-b082-e78d8ddf193e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:29.850152 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-21e97af1-bf94-4f0b-b90e-9219c9f5e3cc admin admin] The token cache is being invalidate because project 82c8133fb930408caf9fa6f10d489cc4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:11:29.850444 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-21e97af1-bf94-4f0b-b90e-9219c9f5e3cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:29.851375 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-21e97af1-bf94-4f0b-b90e-9219c9f5e3cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:29.853382 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1046/2091] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:11:29 2022] DELETE /identity/v3/projects/82c8133fb930408caf9fa6f10d489cc4 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:11:29.857505 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e97af1-bf94-4f0b-b90e-9219c9f5e3cc admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:29.858944 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-21e97af1-bf94-4f0b-b90e-9219c9f5e3cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:29.904636 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a670c941-44d5-459e-b082-e78d8ddf193e None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:29.908263 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2a88d7-fa95-4057-9152-7fb240e578d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa71c71de263478da234f3c5716a6e5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:29.908911 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2a88d7-fa95-4057-9152-7fb240e578d8 service cinder] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:29.914389 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2a88d7-fa95-4057-9152-7fb240e578d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:29.914389 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2a88d7-fa95-4057-9152-7fb240e578d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:29.914389 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-5c2a88d7-fa95-4057-9152-7fb240e578d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:29.924188 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2a88d7-fa95-4057-9152-7fb240e578d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:29.925823 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2a88d7-fa95-4057-9152-7fb240e578d8 service cinder] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:29.947180 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1046/2092] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:11:29 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 22:11:29.947305 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e97af1-bf94-4f0b-b90e-9219c9f5e3cc admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:30.047299 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d786a8-d48e-4e05-a37a-f98c84f0823e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:30.047951 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d786a8-d48e-4e05-a37a-f98c84f0823e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:30.048228 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d786a8-d48e-4e05-a37a-f98c84f0823e admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:30.048527 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d786a8-d48e-4e05-a37a-f98c84f0823e admin admin] PATH_INFO: `/v3/users/09a4f22d75e5494fb014116491a76aaf` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:30.049940 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d786a8-d48e-4e05-a37a-f98c84f0823e admin admin] RBAC: Authorizing `identity:delete_user(user_id=09a4f22d75e5494fb014116491a76aaf)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:30.052834 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d786a8-d48e-4e05-a37a-f98c84f0823e admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:30.311176 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-c4d786a8-d48e-4e05-a37a-f98c84f0823e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09a4f22d75e5494fb014116491a76aaf'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:30.322890 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-c4d786a8-d48e-4e05-a37a-f98c84f0823e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09a4f22d75e5494fb014116491a76aaf'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:30.429049 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-c4d786a8-d48e-4e05-a37a-f98c84f0823e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09a4f22d75e5494fb014116491a76aaf'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:30.437604 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-c4d786a8-d48e-4e05-a37a-f98c84f0823e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09a4f22d75e5494fb014116491a76aaf'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:30.438263 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-c4d786a8-d48e-4e05-a37a-f98c84f0823e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09a4f22d75e5494fb014116491a76aaf'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:30.444267 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1047/2093] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:11:29 2022] DELETE /identity/v3/users/09a4f22d75e5494fb014116491a76aaf => generated 0 bytes in 587 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 22:11:30.720579 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:30.901646 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2a88d7-fa95-4057-9152-7fb240e578d8 service cinder] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:30.913946 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2a88d7-fa95-4057-9152-7fb240e578d8 service cinder] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:30.976393 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946d03da-4682-4eac-bf87-b662713cfd0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:30.977691 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946d03da-4682-4eac-bf87-b662713cfd0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:30.978190 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946d03da-4682-4eac-bf87-b662713cfd0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:30.979556 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:30.980002 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946d03da-4682-4eac-bf87-b662713cfd0e admin admin] PATH_INFO: `/v3/projects/ed7e98a4193146df8f32455ecd6c362f` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:30.981414 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946d03da-4682-4eac-bf87-b662713cfd0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed7e98a4193146df8f32455ecd6c362f)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:30.983469 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946d03da-4682-4eac-bf87-b662713cfd0e admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:31.138262 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-946d03da-4682-4eac-bf87-b662713cfd0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed7e98a4193146df8f32455ecd6c362f', 'request_id': 'req-946d03da-4682-4eac-bf87-b662713cfd0e'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:31.210607 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-946d03da-4682-4eac-bf87-b662713cfd0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed7e98a4193146df8f32455ecd6c362f', 'request_id': 'req-946d03da-4682-4eac-bf87-b662713cfd0e'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:31.217686 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-946d03da-4682-4eac-bf87-b662713cfd0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed7e98a4193146df8f32455ecd6c362f', 'request_id': 'req-946d03da-4682-4eac-bf87-b662713cfd0e'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:31.230018 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-946d03da-4682-4eac-bf87-b662713cfd0e admin admin] The token cache is being invalidate because project ed7e98a4193146df8f32455ecd6c362f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:11:31.231461 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-946d03da-4682-4eac-bf87-b662713cfd0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:31.231461 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-946d03da-4682-4eac-bf87-b662713cfd0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:31.233211 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1047/2094] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:11:30 2022] DELETE /identity/v3/projects/ed7e98a4193146df8f32455ecd6c362f => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:11:31.238190 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629592a4-95db-4def-b1eb-92169de4bac5 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:31.238758 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629592a4-95db-4def-b1eb-92169de4bac5 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:31.238874 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629592a4-95db-4def-b1eb-92169de4bac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:31.260850 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-629592a4-95db-4def-b1eb-92169de4bac5 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:11:31.262749 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-629592a4-95db-4def-b1eb-92169de4bac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:31.328511 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1048/2095] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:11:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:11:31.337855 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b4ebd0-34d1-4f5f-8e33-1bd075e69c61 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:31.338226 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b4ebd0-34d1-4f5f-8e33-1bd075e69c61 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:31.338563 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b4ebd0-34d1-4f5f-8e33-1bd075e69c61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:31.380815 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-a7b4ebd0-34d1-4f5f-8e33-1bd075e69c61 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:11:31.382036 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-a7b4ebd0-34d1-4f5f-8e33-1bd075e69c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:31.391597 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1048/2096] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:11:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:11:31.395983 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629592a4-95db-4def-b1eb-92169de4bac5 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:31.406242 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629592a4-95db-4def-b1eb-92169de4bac5 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:31.410939 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82525607-008b-40cc-af55-f7a7f3ef17bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:31.411537 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82525607-008b-40cc-af55-f7a7f3ef17bc admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:31.411905 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82525607-008b-40cc-af55-f7a7f3ef17bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:31.412174 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82525607-008b-40cc-af55-f7a7f3ef17bc admin admin] PATH_INFO: `/v3/domains` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:31.412883 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82525607-008b-40cc-af55-f7a7f3ef17bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 22:11:31.413182 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82525607-008b-40cc-af55-f7a7f3ef17bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:31.414762 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82525607-008b-40cc-af55-f7a7f3ef17bc admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:31.427242 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1049/2097] 10.222.0.18 () {58 vars in 1153 bytes} [Sat Aug 6 22:11:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 22:11:31.439052 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b4ebd0-34d1-4f5f-8e33-1bd075e69c61 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:31.448148 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b4ebd0-34d1-4f5f-8e33-1bd075e69c61 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:31.453648 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c85107-f83b-4a26-a882-026b40a349ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:31.454384 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c85107-f83b-4a26-a882-026b40a349ef admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:31.454684 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c85107-f83b-4a26-a882-026b40a349ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:31.454939 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c85107-f83b-4a26-a882-026b40a349ef admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:31.455560 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c85107-f83b-4a26-a882-026b40a349ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:31.457596 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c85107-f83b-4a26-a882-026b40a349ef admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:31.567848 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1049/2098] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:11:31 2022] POST /identity/v3/projects => generated 371 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:11:31.571836 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82525607-008b-40cc-af55-f7a7f3ef17bc admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:31.582828 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82525607-008b-40cc-af55-f7a7f3ef17bc admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:31.638787 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48abbb83-2b08-4b84-a28a-aa69f919757b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:31.639395 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48abbb83-2b08-4b84-a28a-aa69f919757b admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:31.639681 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48abbb83-2b08-4b84-a28a-aa69f919757b admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:31.639954 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48abbb83-2b08-4b84-a28a-aa69f919757b admin admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:31.640623 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48abbb83-2b08-4b84-a28a-aa69f919757b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:31.642554 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48abbb83-2b08-4b84-a28a-aa69f919757b admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:31.762191 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1050/2099] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:11:31 2022] POST /identity/v3/users => generated 286 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:11:31.766478 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c85107-f83b-4a26-a882-026b40a349ef admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:31.779583 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c85107-f83b-4a26-a882-026b40a349ef admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:31.786595 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7329d91b-d82e-40d5-a9c7-650c7d04e24b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:31.787285 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7329d91b-d82e-40d5-a9c7-650c7d04e24b admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:31.787564 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7329d91b-d82e-40d5-a9c7-650c7d04e24b admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:31.787834 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7329d91b-d82e-40d5-a9c7-650c7d04e24b admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:31.788506 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7329d91b-d82e-40d5-a9c7-650c7d04e24b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:31.790255 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7329d91b-d82e-40d5-a9c7-650c7d04e24b admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:31.805695 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1050/2100] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:11:31 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:11:31.806857 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48abbb83-2b08-4b84-a28a-aa69f919757b admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:31.822041 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48abbb83-2b08-4b84-a28a-aa69f919757b admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:31.822656 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad71c1d-3fbf-4ba8-9e5f-0e3390ba0db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:31.823307 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad71c1d-3fbf-4ba8-9e5f-0e3390ba0db7 admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:31.823717 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad71c1d-3fbf-4ba8-9e5f-0e3390ba0db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:31.824577 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad71c1d-3fbf-4ba8-9e5f-0e3390ba0db7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:31.825394 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad71c1d-3fbf-4ba8-9e5f-0e3390ba0db7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:31.827051 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad71c1d-3fbf-4ba8-9e5f-0e3390ba0db7 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:31.839232 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1051/2101] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:11:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:11:31.843849 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7329d91b-d82e-40d5-a9c7-650c7d04e24b admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:31.856721 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7329d91b-d82e-40d5-a9c7-650c7d04e24b admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:31.862434 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac5dde3-da4f-401d-a189-444b341b1818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:31.867225 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac5dde3-da4f-401d-a189-444b341b1818 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:31.867516 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac5dde3-da4f-401d-a189-444b341b1818 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:31.867865 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac5dde3-da4f-401d-a189-444b341b1818 admin admin] PATH_INFO: `/v3/projects/eccde3ba1afd475cad4642ae0c45c0d8/users/5c15badd48854b8fabff34e701dd130b/roles/56e4a96094e9434cbbafb22579c034f5` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:31.884181 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac5dde3-da4f-401d-a189-444b341b1818 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eccde3ba1afd475cad4642ae0c45c0d8, user_id=5c15badd48854b8fabff34e701dd130b, role_id=56e4a96094e9434cbbafb22579c034f5)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:31.887122 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac5dde3-da4f-401d-a189-444b341b1818 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:31.903221 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:32.125450 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:32.129065 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1051/2102] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:11:31 2022] PUT /identity/v3/projects/eccde3ba1afd475cad4642ae0c45c0d8/users/5c15badd48854b8fabff34e701dd130b/roles/56e4a96094e9434cbbafb22579c034f5 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:11:32.139889 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c1c6e9-783e-4d89-81f0-52aef22295b7 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:32.140288 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c1c6e9-783e-4d89-81f0-52aef22295b7 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:32.140693 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c1c6e9-783e-4d89-81f0-52aef22295b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:32.171436 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-77c1c6e9-783e-4d89-81f0-52aef22295b7 None None] MFA Rules not processed for user `5c15badd48854b8fabff34e701dd130b`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:11:32.172680 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-77c1c6e9-783e-4d89-81f0-52aef22295b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:32.236415 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1052/2103] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:11:32 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:11:32.247687 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac5dde3-da4f-401d-a189-444b341b1818 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:32.249361 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-7ac5dde3-da4f-401d-a189-444b341b1818 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:32.313641 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac5dde3-da4f-401d-a189-444b341b1818 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:32.318972 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3d26c8-3f9b-4e4d-985a-08b5b457536a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa71c71de263478da234f3c5716a6e5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:32.319706 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3d26c8-3f9b-4e4d-985a-08b5b457536a service cinder] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:32.320115 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3d26c8-3f9b-4e4d-985a-08b5b457536a service cinder] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:32.320392 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3d26c8-3f9b-4e4d-985a-08b5b457536a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:32.341615 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3d26c8-3f9b-4e4d-985a-08b5b457536a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:32.343763 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3d26c8-3f9b-4e4d-985a-08b5b457536a service cinder] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:32.356438 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1052/2104] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:11:32 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:11:32.379168 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c1c6e9-783e-4d89-81f0-52aef22295b7 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:32.446506 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c1c6e9-783e-4d89-81f0-52aef22295b7 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:32.451327 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417262fb-9c31-4f3a-ba46-420195b65ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:32.452034 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417262fb-9c31-4f3a-ba46-420195b65ef1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:32.452458 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417262fb-9c31-4f3a-ba46-420195b65ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:32.452800 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417262fb-9c31-4f3a-ba46-420195b65ef1 admin admin] PATH_INFO: `/v3/users/5c15badd48854b8fabff34e701dd130b` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:32.454046 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417262fb-9c31-4f3a-ba46-420195b65ef1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c15badd48854b8fabff34e701dd130b)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:32.456036 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417262fb-9c31-4f3a-ba46-420195b65ef1 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:32.757160 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-417262fb-9c31-4f3a-ba46-420195b65ef1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c15badd48854b8fabff34e701dd130b'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:32.772170 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-417262fb-9c31-4f3a-ba46-420195b65ef1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c15badd48854b8fabff34e701dd130b'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:32.908698 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-417262fb-9c31-4f3a-ba46-420195b65ef1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c15badd48854b8fabff34e701dd130b'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:32.924065 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-417262fb-9c31-4f3a-ba46-420195b65ef1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c15badd48854b8fabff34e701dd130b'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:32.924065 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-417262fb-9c31-4f3a-ba46-420195b65ef1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c15badd48854b8fabff34e701dd130b'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:32.924065 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1053/2105] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:11:32 2022] DELETE /identity/v3/users/5c15badd48854b8fabff34e701dd130b => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:11:32.928844 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce862930-2f84-4497-883b-0582f2feabb2 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:32.929314 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce862930-2f84-4497-883b-0582f2feabb2 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:32.929468 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce862930-2f84-4497-883b-0582f2feabb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:32.959410 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-ce862930-2f84-4497-883b-0582f2feabb2 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:11:32.960725 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-ce862930-2f84-4497-883b-0582f2feabb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:33.035820 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1053/2106] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:11:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:11:33.046535 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417262fb-9c31-4f3a-ba46-420195b65ef1 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:33.047571 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-417262fb-9c31-4f3a-ba46-420195b65ef1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:33.107937 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417262fb-9c31-4f3a-ba46-420195b65ef1 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:33.112315 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bb67ef-055e-49c3-80ca-898128f9110d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:33.113095 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb67ef-055e-49c3-80ca-898128f9110d service neutron] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:33.113304 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb67ef-055e-49c3-80ca-898128f9110d service neutron] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:33.113576 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb67ef-055e-49c3-80ca-898128f9110d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:33.124306 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bb67ef-055e-49c3-80ca-898128f9110d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:33.125708 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bb67ef-055e-49c3-80ca-898128f9110d service neutron] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:33.135527 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1054/2107] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:11:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 22:11:33.538289 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:33.548083 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce862930-2f84-4497-883b-0582f2feabb2 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:33.549876 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-ce862930-2f84-4497-883b-0582f2feabb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:33.573983 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce862930-2f84-4497-883b-0582f2feabb2 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:33.590260 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f2aed4-28ee-455b-b6fc-c971b2a5b19a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:33.591436 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f2aed4-28ee-455b-b6fc-c971b2a5b19a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:33.591969 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f2aed4-28ee-455b-b6fc-c971b2a5b19a admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:33.592436 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f2aed4-28ee-455b-b6fc-c971b2a5b19a admin admin] PATH_INFO: `/v3/projects/eccde3ba1afd475cad4642ae0c45c0d8` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:33.594024 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f2aed4-28ee-455b-b6fc-c971b2a5b19a admin admin] RBAC: Authorizing `identity:delete_project(project_id=eccde3ba1afd475cad4642ae0c45c0d8)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:33.596471 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f2aed4-28ee-455b-b6fc-c971b2a5b19a admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:33.723434 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-86f2aed4-28ee-455b-b6fc-c971b2a5b19a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eccde3ba1afd475cad4642ae0c45c0d8', 'request_id': 'req-86f2aed4-28ee-455b-b6fc-c971b2a5b19a'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:33.802487 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-86f2aed4-28ee-455b-b6fc-c971b2a5b19a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eccde3ba1afd475cad4642ae0c45c0d8', 'request_id': 'req-86f2aed4-28ee-455b-b6fc-c971b2a5b19a'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:33.813830 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-86f2aed4-28ee-455b-b6fc-c971b2a5b19a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eccde3ba1afd475cad4642ae0c45c0d8', 'request_id': 'req-86f2aed4-28ee-455b-b6fc-c971b2a5b19a'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:33.830682 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-86f2aed4-28ee-455b-b6fc-c971b2a5b19a admin admin] The token cache is being invalidate because project eccde3ba1afd475cad4642ae0c45c0d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:11:33.831176 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-86f2aed4-28ee-455b-b6fc-c971b2a5b19a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:33.832016 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-86f2aed4-28ee-455b-b6fc-c971b2a5b19a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:33.837691 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1054/2108] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:11:33 2022] DELETE /identity/v3/projects/eccde3ba1afd475cad4642ae0c45c0d8 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:11:33.840038 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bb67ef-055e-49c3-80ca-898128f9110d service neutron] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:33.841785 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-d1bb67ef-055e-49c3-80ca-898128f9110d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:33.844116 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125cd9f0-968e-4602-8437-e5688a8a66e4 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:33.844520 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125cd9f0-968e-4602-8437-e5688a8a66e4 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:33.850101 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125cd9f0-968e-4602-8437-e5688a8a66e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:33.857997 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bb67ef-055e-49c3-80ca-898128f9110d service neutron] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:33.883368 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-125cd9f0-968e-4602-8437-e5688a8a66e4 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:11:33.884687 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-125cd9f0-968e-4602-8437-e5688a8a66e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:33.942334 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c69c44-8a22-43da-8006-74734f578167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:33.943246 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c69c44-8a22-43da-8006-74734f578167 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:33.943959 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c69c44-8a22-43da-8006-74734f578167 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:33.944396 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c69c44-8a22-43da-8006-74734f578167 admin admin] PATH_INFO: `/v3/users/b3873412e4fa4f5aae244082c3a196d4` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:33.946202 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c69c44-8a22-43da-8006-74734f578167 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3873412e4fa4f5aae244082c3a196d4)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:33.948121 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c69c44-8a22-43da-8006-74734f578167 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:33.970158 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1055/2109] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:11:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:11:33.978370 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef61cf-272a-47d1-bfce-8062f2bfe864 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:33.978699 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef61cf-272a-47d1-bfce-8062f2bfe864 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:33.979011 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef61cf-272a-47d1-bfce-8062f2bfe864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:34.006393 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-31ef61cf-272a-47d1-bfce-8062f2bfe864 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:11:34.008620 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-31ef61cf-272a-47d1-bfce-8062f2bfe864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:34.020071 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1056/2110] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:11:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 22:11:34.021116 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ef61cf-272a-47d1-bfce-8062f2bfe864 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:34.031160 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ef61cf-272a-47d1-bfce-8062f2bfe864 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:34.036040 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba00f55-ecd9-41ac-8a2b-5c7fb474f5ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:34.036662 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba00f55-ecd9-41ac-8a2b-5c7fb474f5ad admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:34.036942 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba00f55-ecd9-41ac-8a2b-5c7fb474f5ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:34.037211 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba00f55-ecd9-41ac-8a2b-5c7fb474f5ad admin admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:34.037879 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba00f55-ecd9-41ac-8a2b-5c7fb474f5ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 22:11:34.038177 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba00f55-ecd9-41ac-8a2b-5c7fb474f5ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:34.043374 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba00f55-ecd9-41ac-8a2b-5c7fb474f5ad admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:34.052234 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1057/2111] 10.222.0.18 () {58 vars in 1153 bytes} [Sat Aug 6 22:11:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 22:11:34.056563 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba00f55-ecd9-41ac-8a2b-5c7fb474f5ad admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:34.070020 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba00f55-ecd9-41ac-8a2b-5c7fb474f5ad admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:34.075417 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cf20e7-2620-4f13-9820-9e6897f9e745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:34.076045 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cf20e7-2620-4f13-9820-9e6897f9e745 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:34.076322 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cf20e7-2620-4f13-9820-9e6897f9e745 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:34.076630 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cf20e7-2620-4f13-9820-9e6897f9e745 admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:34.077264 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cf20e7-2620-4f13-9820-9e6897f9e745 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:34.079236 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cf20e7-2620-4f13-9820-9e6897f9e745 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:34.159043 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-65c69c44-8a22-43da-8006-74734f578167 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3873412e4fa4f5aae244082c3a196d4'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:34.173521 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-65c69c44-8a22-43da-8006-74734f578167 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3873412e4fa4f5aae244082c3a196d4'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:34.179936 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1058/2112] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:11:34 2022] POST /identity/v3/projects => generated 377 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 22:11:34.185924 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cf20e7-2620-4f13-9820-9e6897f9e745 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:34.251611 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cf20e7-2620-4f13-9820-9e6897f9e745 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:34.256748 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6410117-ccd2-48f2-a09f-a305c25a81b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:34.257357 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6410117-ccd2-48f2-a09f-a305c25a81b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:34.257640 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6410117-ccd2-48f2-a09f-a305c25a81b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:34.257918 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6410117-ccd2-48f2-a09f-a305c25a81b0 admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:34.258547 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6410117-ccd2-48f2-a09f-a305c25a81b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:34.260480 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6410117-ccd2-48f2-a09f-a305c25a81b0 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:34.269321 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-65c69c44-8a22-43da-8006-74734f578167 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3873412e4fa4f5aae244082c3a196d4'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:34.279381 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-65c69c44-8a22-43da-8006-74734f578167 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3873412e4fa4f5aae244082c3a196d4'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:34.279512 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-65c69c44-8a22-43da-8006-74734f578167 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3873412e4fa4f5aae244082c3a196d4'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:34.285717 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1055/2113] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:11:33 2022] DELETE /identity/v3/users/b3873412e4fa4f5aae244082c3a196d4 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:11:34.290732 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9a6c93-eb1e-48fe-a380-888ceb0ed860 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:34.291177 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9a6c93-eb1e-48fe-a380-888ceb0ed860 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:34.291600 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9a6c93-eb1e-48fe-a380-888ceb0ed860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:34.311537 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-cf9a6c93-eb1e-48fe-a380-888ceb0ed860 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:11:34.312883 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-cf9a6c93-eb1e-48fe-a380-888ceb0ed860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:34.375828 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1056/2114] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:11:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 22:11:34.385516 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9a6c93-eb1e-48fe-a380-888ceb0ed860 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:34.387408 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-cf9a6c93-eb1e-48fe-a380-888ceb0ed860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:34.395939 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1059/2115] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:11:34 2022] POST /identity/v3/users => generated 289 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 22:11:34.401330 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6410117-ccd2-48f2-a09f-a305c25a81b0 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:34.403489 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-d6410117-ccd2-48f2-a09f-a305c25a81b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:34.415409 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6410117-ccd2-48f2-a09f-a305c25a81b0 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:34.420334 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af26c0d-d6b7-4ec5-bafa-98e103022477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:34.420950 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af26c0d-d6b7-4ec5-bafa-98e103022477 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:34.421225 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af26c0d-d6b7-4ec5-bafa-98e103022477 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:34.421496 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af26c0d-d6b7-4ec5-bafa-98e103022477 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:34.422162 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af26c0d-d6b7-4ec5-bafa-98e103022477 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:34.423741 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af26c0d-d6b7-4ec5-bafa-98e103022477 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:34.431156 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:34.433831 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1060/2116] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:11:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:11:34.444667 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af26c0d-d6b7-4ec5-bafa-98e103022477 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:34.449197 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9a6c93-eb1e-48fe-a380-888ceb0ed860 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:34.453216 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37a1c3b-75b9-4c4d-9e50-c34140baac31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:34.453978 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a1c3b-75b9-4c4d-9e50-c34140baac31 service neutron] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:34.454265 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a1c3b-75b9-4c4d-9e50-c34140baac31 service neutron] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:34.454538 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a1c3b-75b9-4c4d-9e50-c34140baac31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:34.454639 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af26c0d-d6b7-4ec5-bafa-98e103022477 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:34.459010 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea69476-3b29-4897-a218-4dd368c01ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:34.459585 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea69476-3b29-4897-a218-4dd368c01ff1 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:34.459859 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea69476-3b29-4897-a218-4dd368c01ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:34.460125 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea69476-3b29-4897-a218-4dd368c01ff1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:34.460785 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea69476-3b29-4897-a218-4dd368c01ff1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:34.462351 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea69476-3b29-4897-a218-4dd368c01ff1 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:34.463648 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37a1c3b-75b9-4c4d-9e50-c34140baac31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:34.466044 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37a1c3b-75b9-4c4d-9e50-c34140baac31 service neutron] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:34.473334 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1061/2117] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:11:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 22:11:34.476119 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1057/2118] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:11:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 22:11:34.477390 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea69476-3b29-4897-a218-4dd368c01ff1 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:34.486997 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea69476-3b29-4897-a218-4dd368c01ff1 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:34.491941 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dd1ceb-3832-492f-bf89-98c08e7d00a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:34.492640 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd1ceb-3832-492f-bf89-98c08e7d00a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:34.492951 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd1ceb-3832-492f-bf89-98c08e7d00a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:34.493223 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd1ceb-3832-492f-bf89-98c08e7d00a1 admin admin] PATH_INFO: `/v3/projects/0a6a5feaec104d728c2d182a36a683c5/users/1f36a772b21e4114b160d2b51e7e6825/roles/56e4a96094e9434cbbafb22579c034f5` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:34.508494 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dd1ceb-3832-492f-bf89-98c08e7d00a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a6a5feaec104d728c2d182a36a683c5, user_id=1f36a772b21e4114b160d2b51e7e6825, role_id=56e4a96094e9434cbbafb22579c034f5)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:34.511158 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dd1ceb-3832-492f-bf89-98c08e7d00a1 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:34.577997 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1062/2119] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:11:34 2022] PUT /identity/v3/projects/0a6a5feaec104d728c2d182a36a683c5/users/1f36a772b21e4114b160d2b51e7e6825/roles/56e4a96094e9434cbbafb22579c034f5 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:11:34.592264 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac75b74-53d1-46a9-bd81-0e7cab54b244 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:34.592586 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac75b74-53d1-46a9-bd81-0e7cab54b244 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:34.592860 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac75b74-53d1-46a9-bd81-0e7cab54b244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:34.621165 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-fac75b74-53d1-46a9-bd81-0e7cab54b244 None None] MFA Rules not processed for user `1f36a772b21e4114b160d2b51e7e6825`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:11:34.622362 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-fac75b74-53d1-46a9-bd81-0e7cab54b244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:34.698160 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1058/2120] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:11:34 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:11:34.706269 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dd1ceb-3832-492f-bf89-98c08e7d00a1 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:34.711784 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-b4dd1ceb-3832-492f-bf89-98c08e7d00a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:34.775631 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dd1ceb-3832-492f-bf89-98c08e7d00a1 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:34.779919 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100c7dde-4c39-4d26-a710-b64380f32020 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa71c71de263478da234f3c5716a6e5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:34.780556 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c7dde-4c39-4d26-a710-b64380f32020 service cinder] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:34.780865 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c7dde-4c39-4d26-a710-b64380f32020 service cinder] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:34.781138 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c7dde-4c39-4d26-a710-b64380f32020 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:34.789916 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100c7dde-4c39-4d26-a710-b64380f32020 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:34.791585 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100c7dde-4c39-4d26-a710-b64380f32020 service cinder] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:34.801299 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1063/2121] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:11:34 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:11:35.041781 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac75b74-53d1-46a9-bd81-0e7cab54b244 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:35.044292 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-fac75b74-53d1-46a9-bd81-0e7cab54b244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:35.096439 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:35.131654 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac75b74-53d1-46a9-bd81-0e7cab54b244 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:35.144421 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac725c9-0d15-44bb-8a66-7dda4ab54afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:35.145243 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac725c9-0d15-44bb-8a66-7dda4ab54afe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:35.145525 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac725c9-0d15-44bb-8a66-7dda4ab54afe admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:35.145807 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac725c9-0d15-44bb-8a66-7dda4ab54afe admin admin] PATH_INFO: `/v3/projects/0d7cfb6fbdb94c16930bdf47ce8f8f47` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:35.147031 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac725c9-0d15-44bb-8a66-7dda4ab54afe admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d7cfb6fbdb94c16930bdf47ce8f8f47)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:35.149286 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac725c9-0d15-44bb-8a66-7dda4ab54afe admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:35.183461 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:35.235165 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-6ac725c9-0d15-44bb-8a66-7dda4ab54afe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d7cfb6fbdb94c16930bdf47ce8f8f47', 'request_id': 'req-6ac725c9-0d15-44bb-8a66-7dda4ab54afe'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:35.391199 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-6ac725c9-0d15-44bb-8a66-7dda4ab54afe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d7cfb6fbdb94c16930bdf47ce8f8f47', 'request_id': 'req-6ac725c9-0d15-44bb-8a66-7dda4ab54afe'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:35.406142 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-6ac725c9-0d15-44bb-8a66-7dda4ab54afe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d7cfb6fbdb94c16930bdf47ce8f8f47', 'request_id': 'req-6ac725c9-0d15-44bb-8a66-7dda4ab54afe'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:35.419129 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-6ac725c9-0d15-44bb-8a66-7dda4ab54afe admin admin] The token cache is being invalidate because project 0d7cfb6fbdb94c16930bdf47ce8f8f47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:11:35.419727 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-6ac725c9-0d15-44bb-8a66-7dda4ab54afe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:35.420636 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-6ac725c9-0d15-44bb-8a66-7dda4ab54afe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:35.422436 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1059/2122] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:11:35 2022] DELETE /identity/v3/projects/0d7cfb6fbdb94c16930bdf47ce8f8f47 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:11:35.426463 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100c7dde-4c39-4d26-a710-b64380f32020 service cinder] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:35.438580 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100c7dde-4c39-4d26-a710-b64380f32020 service cinder] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:35.494001 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e85f66d-2164-48ef-a5b4-2401d1a78f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:35.494620 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e85f66d-2164-48ef-a5b4-2401d1a78f98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:35.494905 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e85f66d-2164-48ef-a5b4-2401d1a78f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:35.495177 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e85f66d-2164-48ef-a5b4-2401d1a78f98 admin admin] PATH_INFO: `/v3/users/7709b37725a74594af5181bc1db8bf8c` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:35.497016 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e85f66d-2164-48ef-a5b4-2401d1a78f98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7709b37725a74594af5181bc1db8bf8c)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:35.498987 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e85f66d-2164-48ef-a5b4-2401d1a78f98 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:35.882971 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-0e85f66d-2164-48ef-a5b4-2401d1a78f98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7709b37725a74594af5181bc1db8bf8c'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:35.896021 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-0e85f66d-2164-48ef-a5b4-2401d1a78f98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7709b37725a74594af5181bc1db8bf8c'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:35.902372 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-0e85f66d-2164-48ef-a5b4-2401d1a78f98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7709b37725a74594af5181bc1db8bf8c'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:35.903088 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-0e85f66d-2164-48ef-a5b4-2401d1a78f98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7709b37725a74594af5181bc1db8bf8c'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:35.903713 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-0e85f66d-2164-48ef-a5b4-2401d1a78f98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7709b37725a74594af5181bc1db8bf8c'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:36.018908 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1064/2123] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:11:35 2022] DELETE /identity/v3/users/7709b37725a74594af5181bc1db8bf8c => generated 0 bytes in 593 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:11:36.504081 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac725c9-0d15-44bb-8a66-7dda4ab54afe admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:36.505662 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-6ac725c9-0d15-44bb-8a66-7dda4ab54afe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:36.519013 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac725c9-0d15-44bb-8a66-7dda4ab54afe admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:36.575546 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34739475-77a7-4ab1-a554-823c5e85f07e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:36.576111 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34739475-77a7-4ab1-a554-823c5e85f07e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:36.576387 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34739475-77a7-4ab1-a554-823c5e85f07e admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:36.576691 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34739475-77a7-4ab1-a554-823c5e85f07e admin admin] PATH_INFO: `/v3/projects/0574f5cee7d54d0d9b2a3fe259babda5` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:36.577827 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34739475-77a7-4ab1-a554-823c5e85f07e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0574f5cee7d54d0d9b2a3fe259babda5)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:36.579726 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34739475-77a7-4ab1-a554-823c5e85f07e admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:36.691552 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-34739475-77a7-4ab1-a554-823c5e85f07e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0574f5cee7d54d0d9b2a3fe259babda5', 'request_id': 'req-34739475-77a7-4ab1-a554-823c5e85f07e'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:36.771128 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-34739475-77a7-4ab1-a554-823c5e85f07e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0574f5cee7d54d0d9b2a3fe259babda5', 'request_id': 'req-34739475-77a7-4ab1-a554-823c5e85f07e'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:36.777821 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-34739475-77a7-4ab1-a554-823c5e85f07e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0574f5cee7d54d0d9b2a3fe259babda5', 'request_id': 'req-34739475-77a7-4ab1-a554-823c5e85f07e'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:36.788938 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-34739475-77a7-4ab1-a554-823c5e85f07e admin admin] The token cache is being invalidate because project 0574f5cee7d54d0d9b2a3fe259babda5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:11:36.789222 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-34739475-77a7-4ab1-a554-823c5e85f07e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:36.789894 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-34739475-77a7-4ab1-a554-823c5e85f07e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:36.791438 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1060/2124] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:11:36 2022] DELETE /identity/v3/projects/0574f5cee7d54d0d9b2a3fe259babda5 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:11:36.796873 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3619da5-57ac-4e7b-a950-d16299f4315e None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:36.797358 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3619da5-57ac-4e7b-a950-d16299f4315e None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:36.797467 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3619da5-57ac-4e7b-a950-d16299f4315e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:36.816095 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-f3619da5-57ac-4e7b-a950-d16299f4315e None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:11:36.817319 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-f3619da5-57ac-4e7b-a950-d16299f4315e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:36.876337 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1065/2125] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:11:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:11:36.883809 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0de996-bff4-4935-83bd-47dc7a16f872 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:36.884090 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0de996-bff4-4935-83bd-47dc7a16f872 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:36.884361 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0de996-bff4-4935-83bd-47dc7a16f872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:36.905455 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-ca0de996-bff4-4935-83bd-47dc7a16f872 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:11:36.906533 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-ca0de996-bff4-4935-83bd-47dc7a16f872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:36.914376 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1061/2126] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:11:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:11:36.919354 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3619da5-57ac-4e7b-a950-d16299f4315e None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:36.930118 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3619da5-57ac-4e7b-a950-d16299f4315e None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:36.934848 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39783772-e752-43db-af81-8cad7b840873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:36.935444 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39783772-e752-43db-af81-8cad7b840873 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:36.935720 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39783772-e752-43db-af81-8cad7b840873 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:36.935989 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39783772-e752-43db-af81-8cad7b840873 admin admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:36.936655 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39783772-e752-43db-af81-8cad7b840873 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 22:11:36.936963 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39783772-e752-43db-af81-8cad7b840873 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:36.938533 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39783772-e752-43db-af81-8cad7b840873 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:36.950372 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1066/2127] 10.222.0.18 () {58 vars in 1153 bytes} [Sat Aug 6 22:11:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 22:11:36.959923 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0de996-bff4-4935-83bd-47dc7a16f872 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:36.970234 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0de996-bff4-4935-83bd-47dc7a16f872 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:36.974891 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f59eee4-2132-4fa7-a717-f36d0f91cb5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:36.975489 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f59eee4-2132-4fa7-a717-f36d0f91cb5e admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:36.976057 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f59eee4-2132-4fa7-a717-f36d0f91cb5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:36.976352 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f59eee4-2132-4fa7-a717-f36d0f91cb5e admin admin] PATH_INFO: `/v3/projects` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:36.977023 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f59eee4-2132-4fa7-a717-f36d0f91cb5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:36.979467 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f59eee4-2132-4fa7-a717-f36d0f91cb5e admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:36.984984 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:37.006138 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:37.104921 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1062/2128] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:11:36 2022] POST /identity/v3/projects => generated 371 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:11:37.109168 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39783772-e752-43db-af81-8cad7b840873 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:37.119260 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39783772-e752-43db-af81-8cad7b840873 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:37.176227 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b489c4-6065-4d5b-9571-7860fb8de684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:37.176853 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b489c4-6065-4d5b-9571-7860fb8de684 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:37.177132 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b489c4-6065-4d5b-9571-7860fb8de684 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:37.177402 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b489c4-6065-4d5b-9571-7860fb8de684 admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:37.178024 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b489c4-6065-4d5b-9571-7860fb8de684 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:37.179923 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b489c4-6065-4d5b-9571-7860fb8de684 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:37.263610 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1067/2129] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:11:37 2022] POST /identity/v3/users => generated 286 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:11:37.274935 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f59eee4-2132-4fa7-a717-f36d0f91cb5e admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:37.286219 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f59eee4-2132-4fa7-a717-f36d0f91cb5e admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:37.291201 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c4cd59-2242-495d-9aef-300ab8b81e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:37.291818 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c4cd59-2242-495d-9aef-300ab8b81e0c admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:37.292102 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c4cd59-2242-495d-9aef-300ab8b81e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:37.292440 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c4cd59-2242-495d-9aef-300ab8b81e0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:37.293268 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c4cd59-2242-495d-9aef-300ab8b81e0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:37.294885 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c4cd59-2242-495d-9aef-300ab8b81e0c admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:37.305972 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1063/2130] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:11:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:11:37.318441 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b489c4-6065-4d5b-9571-7860fb8de684 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:37.329484 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b489c4-6065-4d5b-9571-7860fb8de684 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:37.334249 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bc367d-c223-4dae-89c8-a466644f6c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:37.334840 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bc367d-c223-4dae-89c8-a466644f6c3f admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:37.335119 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bc367d-c223-4dae-89c8-a466644f6c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:37.335397 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bc367d-c223-4dae-89c8-a466644f6c3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:37.336033 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bc367d-c223-4dae-89c8-a466644f6c3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:37.337677 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bc367d-c223-4dae-89c8-a466644f6c3f admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:37.349637 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1068/2131] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:11:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:11:37.353693 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c4cd59-2242-495d-9aef-300ab8b81e0c admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:37.364451 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c4cd59-2242-495d-9aef-300ab8b81e0c admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:37.368833 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2401547-98a1-43ab-9386-e3b839c26f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:37.369562 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2401547-98a1-43ab-9386-e3b839c26f61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:37.369815 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2401547-98a1-43ab-9386-e3b839c26f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:37.370093 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2401547-98a1-43ab-9386-e3b839c26f61 admin admin] PATH_INFO: `/v3/projects/6cc6b84f1fcc4e28b7c3a7547b6640b8/users/26b6e4a0640548dd8d6bb800d1f26b95/roles/56e4a96094e9434cbbafb22579c034f5` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:37.385272 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2401547-98a1-43ab-9386-e3b839c26f61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cc6b84f1fcc4e28b7c3a7547b6640b8, user_id=26b6e4a0640548dd8d6bb800d1f26b95, role_id=56e4a96094e9434cbbafb22579c034f5)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:37.388465 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2401547-98a1-43ab-9386-e3b839c26f61 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:37.481022 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1064/2132] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:11:37 2022] PUT /identity/v3/projects/6cc6b84f1fcc4e28b7c3a7547b6640b8/users/26b6e4a0640548dd8d6bb800d1f26b95/roles/56e4a96094e9434cbbafb22579c034f5 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:11:37.486669 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed13913-50b0-41b3-9a96-ad181c495ea7 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:37.486947 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed13913-50b0-41b3-9a96-ad181c495ea7 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:37.487220 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed13913-50b0-41b3-9a96-ad181c495ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:37.506671 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-fed13913-50b0-41b3-9a96-ad181c495ea7 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:11:37.508980 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-fed13913-50b0-41b3-9a96-ad181c495ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:37.575015 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1069/2133] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:11:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:11:37.584183 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2401547-98a1-43ab-9386-e3b839c26f61 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:37.590176 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-e2401547-98a1-43ab-9386-e3b839c26f61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:37.646665 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2401547-98a1-43ab-9386-e3b839c26f61 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:37.650314 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c42a6e0-25b9-4858-8081-b73e88469b29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:37.650909 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c42a6e0-25b9-4858-8081-b73e88469b29 service neutron] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:37.651184 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c42a6e0-25b9-4858-8081-b73e88469b29 service neutron] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:37.651450 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c42a6e0-25b9-4858-8081-b73e88469b29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:37.661052 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c42a6e0-25b9-4858-8081-b73e88469b29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:37.662471 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c42a6e0-25b9-4858-8081-b73e88469b29 service neutron] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:37.672744 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1065/2134] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:11:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:11:38.006533 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:38.080461 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:38.423052 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:38.457328 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:38.743410 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:38.824332 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:39.220291 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc4415e-243c-46e2-9efb-fd0b3703dd06 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:39.220867 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc4415e-243c-46e2-9efb-fd0b3703dd06 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:39.220867 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc4415e-243c-46e2-9efb-fd0b3703dd06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:39.249931 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-6bc4415e-243c-46e2-9efb-fd0b3703dd06 None None] MFA Rules not processed for user `26b6e4a0640548dd8d6bb800d1f26b95`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:11:39.251314 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-6bc4415e-243c-46e2-9efb-fd0b3703dd06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:39.276426 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:39.326257 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1070/2135] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:11:39 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 22:11:39.337201 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c42a6e0-25b9-4858-8081-b73e88469b29 service neutron] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:39.339316 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-2c42a6e0-25b9-4858-8081-b73e88469b29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:39.401804 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c42a6e0-25b9-4858-8081-b73e88469b29 service neutron] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:39.406069 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e4b527-7e20-4d31-8b4b-8887e3785f58 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa71c71de263478da234f3c5716a6e5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:39.407865 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e4b527-7e20-4d31-8b4b-8887e3785f58 service cinder] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:39.407865 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e4b527-7e20-4d31-8b4b-8887e3785f58 service cinder] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:39.407865 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e4b527-7e20-4d31-8b4b-8887e3785f58 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:39.418458 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e4b527-7e20-4d31-8b4b-8887e3785f58 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:39.419927 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e4b527-7e20-4d31-8b4b-8887e3785f58 service cinder] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:39.429543 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1066/2136] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:11:39 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:11:39.440569 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:39.966846 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:40.666253 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:41.011868 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:41.451714 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:41.653009 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:41.858778 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc4415e-243c-46e2-9efb-fd0b3703dd06 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:41.860853 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-6bc4415e-243c-46e2-9efb-fd0b3703dd06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:41.878155 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc4415e-243c-46e2-9efb-fd0b3703dd06 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:41.888015 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a649569-c5f1-46e2-90ab-8683caf80cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:41.888851 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a649569-c5f1-46e2-90ab-8683caf80cf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:41.889238 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a649569-c5f1-46e2-90ab-8683caf80cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:41.889989 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a649569-c5f1-46e2-90ab-8683caf80cf9 admin admin] PATH_INFO: `/v3/users/d20a99eab400474c88f3e42754d14080` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:41.891454 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a649569-c5f1-46e2-90ab-8683caf80cf9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d20a99eab400474c88f3e42754d14080)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:41.893644 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a649569-c5f1-46e2-90ab-8683caf80cf9 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:42.137186 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-4a649569-c5f1-46e2-90ab-8683caf80cf9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd20a99eab400474c88f3e42754d14080'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:42.156587 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-4a649569-c5f1-46e2-90ab-8683caf80cf9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd20a99eab400474c88f3e42754d14080'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:42.162902 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-4a649569-c5f1-46e2-90ab-8683caf80cf9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd20a99eab400474c88f3e42754d14080'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:42.163668 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-4a649569-c5f1-46e2-90ab-8683caf80cf9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd20a99eab400474c88f3e42754d14080'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:42.164302 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-4a649569-c5f1-46e2-90ab-8683caf80cf9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd20a99eab400474c88f3e42754d14080'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:42.274270 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1071/2137] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:11:41 2022] DELETE /identity/v3/users/d20a99eab400474c88f3e42754d14080 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:11:42.279480 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacf0535-507c-499d-82a7-10d2d16455db None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:42.280003 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacf0535-507c-499d-82a7-10d2d16455db None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:42.280343 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacf0535-507c-499d-82a7-10d2d16455db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:42.290142 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a649569-c5f1-46e2-90ab-8683caf80cf9 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:42.291910 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-4a649569-c5f1-46e2-90ab-8683caf80cf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:42.301712 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-dacf0535-507c-499d-82a7-10d2d16455db None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:11:42.302925 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-dacf0535-507c-499d-82a7-10d2d16455db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:42.394517 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1067/2138] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:11:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 22:11:42.405130 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a649569-c5f1-46e2-90ab-8683caf80cf9 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:42.407834 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacf0535-507c-499d-82a7-10d2d16455db None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:42.409757 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-dacf0535-507c-499d-82a7-10d2d16455db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:42.411958 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786ca4d0-256c-45d0-b8d5-b00fa10519f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:42.412737 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786ca4d0-256c-45d0-b8d5-b00fa10519f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:42.413175 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786ca4d0-256c-45d0-b8d5-b00fa10519f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:42.413645 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786ca4d0-256c-45d0-b8d5-b00fa10519f2 admin admin] PATH_INFO: `/v3/users/4b065a4aa3424317a646faa80af11016` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:42.414980 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786ca4d0-256c-45d0-b8d5-b00fa10519f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b065a4aa3424317a646faa80af11016)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:42.417100 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786ca4d0-256c-45d0-b8d5-b00fa10519f2 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:42.462964 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:42.493663 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacf0535-507c-499d-82a7-10d2d16455db None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:42.498064 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6c62df-5166-4e89-9f69-158c6d67ce8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:42.498667 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c62df-5166-4e89-9f69-158c6d67ce8b service neutron] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:42.498943 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c62df-5166-4e89-9f69-158c6d67ce8b service neutron] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:42.499229 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c62df-5166-4e89-9f69-158c6d67ce8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:42.508327 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6c62df-5166-4e89-9f69-158c6d67ce8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:42.509794 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6c62df-5166-4e89-9f69-158c6d67ce8b service neutron] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:42.519544 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1068/2139] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:11:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 22:11:42.520843 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29061fb7-a53d-4df3-aca2-e2f2561d357e None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:42.521049 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29061fb7-a53d-4df3-aca2-e2f2561d357e None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:42.521318 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29061fb7-a53d-4df3-aca2-e2f2561d357e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:42.540680 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-29061fb7-a53d-4df3-aca2-e2f2561d357e None None] MFA Rules not processed for user `26b6e4a0640548dd8d6bb800d1f26b95`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:11:42.542825 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-29061fb7-a53d-4df3-aca2-e2f2561d357e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:42.561518 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:42.620288 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1069/2140] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:11:42 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:11:42.651972 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-786ca4d0-256c-45d0-b8d5-b00fa10519f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b065a4aa3424317a646faa80af11016'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:42.665750 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-786ca4d0-256c-45d0-b8d5-b00fa10519f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b065a4aa3424317a646faa80af11016'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:42.677717 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-786ca4d0-256c-45d0-b8d5-b00fa10519f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b065a4aa3424317a646faa80af11016'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:42.678590 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-786ca4d0-256c-45d0-b8d5-b00fa10519f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b065a4aa3424317a646faa80af11016'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:42.679202 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-786ca4d0-256c-45d0-b8d5-b00fa10519f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b065a4aa3424317a646faa80af11016'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:42.961499 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1072/2141] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:11:42 2022] DELETE /identity/v3/users/4b065a4aa3424317a646faa80af11016 => generated 0 bytes in 672 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 22:11:42.967697 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810a0772-00d0-4fb6-a5da-8aa9f0f06c5c None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:42.968059 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810a0772-00d0-4fb6-a5da-8aa9f0f06c5c None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:42.968421 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810a0772-00d0-4fb6-a5da-8aa9f0f06c5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:42.991746 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-810a0772-00d0-4fb6-a5da-8aa9f0f06c5c None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:11:42.993357 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-810a0772-00d0-4fb6-a5da-8aa9f0f06c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:43.055615 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1070/2142] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:11:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:11:43.065222 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786ca4d0-256c-45d0-b8d5-b00fa10519f2 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:43.066716 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-786ca4d0-256c-45d0-b8d5-b00fa10519f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:43.129800 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786ca4d0-256c-45d0-b8d5-b00fa10519f2 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:43.132835 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810a0772-00d0-4fb6-a5da-8aa9f0f06c5c None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:43.133750 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e07bae-6d93-40ed-81f3-3fd12ac8db1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:43.134295 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-810a0772-00d0-4fb6-a5da-8aa9f0f06c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:43.134514 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e07bae-6d93-40ed-81f3-3fd12ac8db1d service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:43.134832 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e07bae-6d93-40ed-81f3-3fd12ac8db1d service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:43.135168 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e07bae-6d93-40ed-81f3-3fd12ac8db1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:43.149117 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e07bae-6d93-40ed-81f3-3fd12ac8db1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:43.150764 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e07bae-6d93-40ed-81f3-3fd12ac8db1d service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:43.157190 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810a0772-00d0-4fb6-a5da-8aa9f0f06c5c None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:43.160419 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:43.168566 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569aaa58-7036-40f8-a677-e66c591c7e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:43.169565 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569aaa58-7036-40f8-a677-e66c591c7e6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:43.169989 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569aaa58-7036-40f8-a677-e66c591c7e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:43.170428 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569aaa58-7036-40f8-a677-e66c591c7e6a admin admin] PATH_INFO: `/v3/projects/3ae1ea2ce54d47c4829ad2ad39d1eb77` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:43.172005 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569aaa58-7036-40f8-a677-e66c591c7e6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ae1ea2ce54d47c4829ad2ad39d1eb77)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:43.174011 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1073/2143] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:11:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:11:43.174369 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569aaa58-7036-40f8-a677-e66c591c7e6a admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:43.375745 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-569aaa58-7036-40f8-a677-e66c591c7e6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ae1ea2ce54d47c4829ad2ad39d1eb77', 'request_id': 'req-569aaa58-7036-40f8-a677-e66c591c7e6a'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:43.446075 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-569aaa58-7036-40f8-a677-e66c591c7e6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ae1ea2ce54d47c4829ad2ad39d1eb77', 'request_id': 'req-569aaa58-7036-40f8-a677-e66c591c7e6a'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:43.458878 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-569aaa58-7036-40f8-a677-e66c591c7e6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ae1ea2ce54d47c4829ad2ad39d1eb77', 'request_id': 'req-569aaa58-7036-40f8-a677-e66c591c7e6a'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:43.471697 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-569aaa58-7036-40f8-a677-e66c591c7e6a admin admin] The token cache is being invalidate because project 3ae1ea2ce54d47c4829ad2ad39d1eb77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:11:43.471995 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-569aaa58-7036-40f8-a677-e66c591c7e6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:43.472751 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-569aaa58-7036-40f8-a677-e66c591c7e6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:43.474380 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1071/2144] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:11:43 2022] DELETE /identity/v3/projects/3ae1ea2ce54d47c4829ad2ad39d1eb77 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 22:11:43.478917 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e07bae-6d93-40ed-81f3-3fd12ac8db1d service neutron] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:43.488898 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e07bae-6d93-40ed-81f3-3fd12ac8db1d service neutron] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:43.629077 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60ce19c-1720-48b6-83e6-799d75d4da54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:43.630025 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ce19c-1720-48b6-83e6-799d75d4da54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:43.630381 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ce19c-1720-48b6-83e6-799d75d4da54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:43.630717 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ce19c-1720-48b6-83e6-799d75d4da54 admin admin] PATH_INFO: `/v3/users/6eda50a5b7a14ccc989be8c46b992990` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:43.651368 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60ce19c-1720-48b6-83e6-799d75d4da54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6eda50a5b7a14ccc989be8c46b992990)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:43.651506 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569aaa58-7036-40f8-a677-e66c591c7e6a admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:43.651506 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-569aaa58-7036-40f8-a677-e66c591c7e6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:43.651712 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60ce19c-1720-48b6-83e6-799d75d4da54 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:43.677206 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569aaa58-7036-40f8-a677-e66c591c7e6a admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:43.719287 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f173dc-c694-4e90-8af4-ef006a01d80c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:43.719287 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f173dc-c694-4e90-8af4-ef006a01d80c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:43.719287 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f173dc-c694-4e90-8af4-ef006a01d80c admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:43.719287 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f173dc-c694-4e90-8af4-ef006a01d80c admin admin] PATH_INFO: `/v3/projects/72806f3bf23a42c58952a0f3b6312ba6` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:43.719599 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f173dc-c694-4e90-8af4-ef006a01d80c admin admin] RBAC: Authorizing `identity:delete_project(project_id=72806f3bf23a42c58952a0f3b6312ba6)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:43.721844 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f173dc-c694-4e90-8af4-ef006a01d80c admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:43.738783 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:44.059587 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-d60ce19c-1720-48b6-83e6-799d75d4da54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6eda50a5b7a14ccc989be8c46b992990'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:44.063339 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-83f173dc-c694-4e90-8af4-ef006a01d80c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72806f3bf23a42c58952a0f3b6312ba6', 'request_id': 'req-83f173dc-c694-4e90-8af4-ef006a01d80c'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:44.076493 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-d60ce19c-1720-48b6-83e6-799d75d4da54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6eda50a5b7a14ccc989be8c46b992990'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:44.085970 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-d60ce19c-1720-48b6-83e6-799d75d4da54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6eda50a5b7a14ccc989be8c46b992990'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:44.086719 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-d60ce19c-1720-48b6-83e6-799d75d4da54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6eda50a5b7a14ccc989be8c46b992990'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:44.087340 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-d60ce19c-1720-48b6-83e6-799d75d4da54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6eda50a5b7a14ccc989be8c46b992990'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:44.154320 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-83f173dc-c694-4e90-8af4-ef006a01d80c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72806f3bf23a42c58952a0f3b6312ba6', 'request_id': 'req-83f173dc-c694-4e90-8af4-ef006a01d80c'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:44.160934 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-83f173dc-c694-4e90-8af4-ef006a01d80c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72806f3bf23a42c58952a0f3b6312ba6', 'request_id': 'req-83f173dc-c694-4e90-8af4-ef006a01d80c'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:44.173679 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-83f173dc-c694-4e90-8af4-ef006a01d80c admin admin] The token cache is being invalidate because project 72806f3bf23a42c58952a0f3b6312ba6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:11:44.173971 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-83f173dc-c694-4e90-8af4-ef006a01d80c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:44.175248 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-83f173dc-c694-4e90-8af4-ef006a01d80c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:44.177023 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1072/2145] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:11:43 2022] DELETE /identity/v3/projects/72806f3bf23a42c58952a0f3b6312ba6 => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:11:44.181512 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd119f4c-9747-4400-9bc3-3aaee565c027 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:44.181838 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd119f4c-9747-4400-9bc3-3aaee565c027 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:44.182117 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd119f4c-9747-4400-9bc3-3aaee565c027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:44.207138 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-bd119f4c-9747-4400-9bc3-3aaee565c027 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:11:44.207138 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-bd119f4c-9747-4400-9bc3-3aaee565c027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:44.257267 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1074/2146] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:11:43 2022] DELETE /identity/v3/users/6eda50a5b7a14ccc989be8c46b992990 => generated 0 bytes in 779 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:11:44.268758 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1073/2147] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:11:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 22:11:44.278921 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e208fa4c-990d-49d3-9b29-9592bc9d89f6 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:44.280957 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e208fa4c-990d-49d3-9b29-9592bc9d89f6 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:44.280957 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e208fa4c-990d-49d3-9b29-9592bc9d89f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:44.318701 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-e208fa4c-990d-49d3-9b29-9592bc9d89f6 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:11:44.320070 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-e208fa4c-990d-49d3-9b29-9592bc9d89f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:44.403991 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1075/2148] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:11:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:11:44.405098 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd119f4c-9747-4400-9bc3-3aaee565c027 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:44.422326 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd119f4c-9747-4400-9bc3-3aaee565c027 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:44.427930 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192c5988-880f-40fc-881d-29a1a86bcf2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:44.428591 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192c5988-880f-40fc-881d-29a1a86bcf2b admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:44.429010 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192c5988-880f-40fc-881d-29a1a86bcf2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:44.429338 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192c5988-880f-40fc-881d-29a1a86bcf2b admin admin] PATH_INFO: `/v3/domains` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:44.430019 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192c5988-880f-40fc-881d-29a1a86bcf2b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 22:11:44.430325 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192c5988-880f-40fc-881d-29a1a86bcf2b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:44.431914 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192c5988-880f-40fc-881d-29a1a86bcf2b admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:44.445117 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1074/2149] 10.222.0.18 () {58 vars in 1153 bytes} [Sat Aug 6 22:11:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 22:11:44.450211 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e208fa4c-990d-49d3-9b29-9592bc9d89f6 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:44.459884 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e208fa4c-990d-49d3-9b29-9592bc9d89f6 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:44.465204 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42005b9a-f896-4e5e-afbf-a9f36fbb1ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:44.465810 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42005b9a-f896-4e5e-afbf-a9f36fbb1ee7 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:44.466113 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42005b9a-f896-4e5e-afbf-a9f36fbb1ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:44.466584 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42005b9a-f896-4e5e-afbf-a9f36fbb1ee7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:44.467258 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42005b9a-f896-4e5e-afbf-a9f36fbb1ee7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:44.469239 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42005b9a-f896-4e5e-afbf-a9f36fbb1ee7 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:44.564638 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1076/2150] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:11:44 2022] POST /identity/v3/projects => generated 381 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:11:44.568608 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192c5988-880f-40fc-881d-29a1a86bcf2b admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:44.582850 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192c5988-880f-40fc-881d-29a1a86bcf2b admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:44.652868 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:44.674402 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab170d24-02bc-478a-b77b-fecd74931d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:44.675015 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab170d24-02bc-478a-b77b-fecd74931d14 admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:44.675893 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab170d24-02bc-478a-b77b-fecd74931d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:44.676166 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab170d24-02bc-478a-b77b-fecd74931d14 admin admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:44.676831 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab170d24-02bc-478a-b77b-fecd74931d14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:44.678741 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab170d24-02bc-478a-b77b-fecd74931d14 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:44.840713 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1075/2151] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:11:44 2022] POST /identity/v3/users => generated 291 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:11:44.844823 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42005b9a-f896-4e5e-afbf-a9f36fbb1ee7 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:44.856846 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42005b9a-f896-4e5e-afbf-a9f36fbb1ee7 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:44.861632 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf980d36-fd82-47e1-b1fa-a824db44481b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:44.862242 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf980d36-fd82-47e1-b1fa-a824db44481b admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:44.862534 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf980d36-fd82-47e1-b1fa-a824db44481b admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:44.862807 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf980d36-fd82-47e1-b1fa-a824db44481b admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:44.863476 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf980d36-fd82-47e1-b1fa-a824db44481b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:44.865123 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf980d36-fd82-47e1-b1fa-a824db44481b admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:44.875667 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1077/2152] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:11:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:11:44.879631 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab170d24-02bc-478a-b77b-fecd74931d14 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:44.890277 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab170d24-02bc-478a-b77b-fecd74931d14 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:44.895682 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcb2f72-6340-4ce4-b5db-ef8cd82d9eea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:44.896377 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcb2f72-6340-4ce4-b5db-ef8cd82d9eea admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:44.896865 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcb2f72-6340-4ce4-b5db-ef8cd82d9eea admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:44.897284 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcb2f72-6340-4ce4-b5db-ef8cd82d9eea admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:44.898084 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcb2f72-6340-4ce4-b5db-ef8cd82d9eea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:44.900157 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf980d36-fd82-47e1-b1fa-a824db44481b admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:44.900333 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcb2f72-6340-4ce4-b5db-ef8cd82d9eea admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:44.903969 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-cf980d36-fd82-47e1-b1fa-a824db44481b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:44.918984 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1076/2153] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:11:44 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:11:44.921010 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf980d36-fd82-47e1-b1fa-a824db44481b admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:44.923185 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcb2f72-6340-4ce4-b5db-ef8cd82d9eea admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:44.927248 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dee31e-7c7a-4723-a2e9-aecc7be12eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:44.927895 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dee31e-7c7a-4723-a2e9-aecc7be12eb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:44.928173 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dee31e-7c7a-4723-a2e9-aecc7be12eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:44.928478 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dee31e-7c7a-4723-a2e9-aecc7be12eb7 admin admin] PATH_INFO: `/v3/projects/a796972f745449b68ae0855452099f1b` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:44.929651 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dee31e-7c7a-4723-a2e9-aecc7be12eb7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a796972f745449b68ae0855452099f1b)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:44.931669 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dee31e-7c7a-4723-a2e9-aecc7be12eb7 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:44.935396 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcb2f72-6340-4ce4-b5db-ef8cd82d9eea admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:44.941294 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54689533-a298-48f0-ab48-3d885309a9ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:44.941294 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54689533-a298-48f0-ab48-3d885309a9ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:44.941294 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54689533-a298-48f0-ab48-3d885309a9ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:44.941565 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54689533-a298-48f0-ab48-3d885309a9ec admin admin] PATH_INFO: `/v3/projects/dc2fb805386743d1a4bfd3b15bde695c/users/56138f43f00e46ea9dda399a0fb0e822/roles/56e4a96094e9434cbbafb22579c034f5` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:44.956681 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54689533-a298-48f0-ab48-3d885309a9ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc2fb805386743d1a4bfd3b15bde695c, user_id=56138f43f00e46ea9dda399a0fb0e822, role_id=56e4a96094e9434cbbafb22579c034f5)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:44.959327 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54689533-a298-48f0-ab48-3d885309a9ec admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:45.095103 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1077/2154] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:11:44 2022] PUT /identity/v3/projects/dc2fb805386743d1a4bfd3b15bde695c/users/56138f43f00e46ea9dda399a0fb0e822/roles/56e4a96094e9434cbbafb22579c034f5 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 22:11:45.101498 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e48300b-d8ec-497f-98fa-97aaa0ec0a38 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:45.101861 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e48300b-d8ec-497f-98fa-97aaa0ec0a38 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:45.102245 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e48300b-d8ec-497f-98fa-97aaa0ec0a38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:45.109822 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-c0dee31e-7c7a-4723-a2e9-aecc7be12eb7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a796972f745449b68ae0855452099f1b', 'request_id': 'req-c0dee31e-7c7a-4723-a2e9-aecc7be12eb7'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:45.125769 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-0e48300b-d8ec-497f-98fa-97aaa0ec0a38 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:11:45.127009 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-0e48300b-d8ec-497f-98fa-97aaa0ec0a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:45.195086 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1078/2155] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:11:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:11:45.205427 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e48300b-d8ec-497f-98fa-97aaa0ec0a38 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:45.206840 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-0e48300b-d8ec-497f-98fa-97aaa0ec0a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:45.211886 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-c0dee31e-7c7a-4723-a2e9-aecc7be12eb7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a796972f745449b68ae0855452099f1b', 'request_id': 'req-c0dee31e-7c7a-4723-a2e9-aecc7be12eb7'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:45.219275 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-c0dee31e-7c7a-4723-a2e9-aecc7be12eb7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a796972f745449b68ae0855452099f1b', 'request_id': 'req-c0dee31e-7c7a-4723-a2e9-aecc7be12eb7'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:45.249929 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-c0dee31e-7c7a-4723-a2e9-aecc7be12eb7 admin admin] The token cache is being invalidate because project a796972f745449b68ae0855452099f1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:11:45.250209 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-c0dee31e-7c7a-4723-a2e9-aecc7be12eb7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:45.250911 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-c0dee31e-7c7a-4723-a2e9-aecc7be12eb7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:45.252452 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1078/2156] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:11:44 2022] DELETE /identity/v3/projects/a796972f745449b68ae0855452099f1b => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:11:45.257463 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66762ad-60c2-4018-901f-da466c858731 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:45.258018 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66762ad-60c2-4018-901f-da466c858731 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:45.258018 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66762ad-60c2-4018-901f-da466c858731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:45.270227 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e48300b-d8ec-497f-98fa-97aaa0ec0a38 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:45.274302 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07830698-f4f9-4b34-9356-4d9c1f3912ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:45.275010 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07830698-f4f9-4b34-9356-4d9c1f3912ca service neutron] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:45.275369 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07830698-f4f9-4b34-9356-4d9c1f3912ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:45.275642 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07830698-f4f9-4b34-9356-4d9c1f3912ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:45.278027 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-07830698-f4f9-4b34-9356-4d9c1f3912ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:45.279199 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-a66762ad-60c2-4018-901f-da466c858731 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:11:45.280394 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-a66762ad-60c2-4018-901f-da466c858731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:45.286230 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:45.291274 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07830698-f4f9-4b34-9356-4d9c1f3912ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:45.293055 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1079/2157] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:11:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:11:45.293726 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07830698-f4f9-4b34-9356-4d9c1f3912ca service neutron] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:45.302467 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe5f68-c515-40ab-bc89-536716ef6f58 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:45.303039 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe5f68-c515-40ab-bc89-536716ef6f58 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:45.303318 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe5f68-c515-40ab-bc89-536716ef6f58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:45.304931 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1079/2158] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:11:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:11:45.325544 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-aabe5f68-c515-40ab-bc89-536716ef6f58 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:11:45.327036 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-aabe5f68-c515-40ab-bc89-536716ef6f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:45.336882 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1080/2159] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:11:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 22:11:45.341812 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07830698-f4f9-4b34-9356-4d9c1f3912ca service neutron] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:45.353487 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07830698-f4f9-4b34-9356-4d9c1f3912ca service neutron] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:45.358590 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6862cc0-f915-4d13-8cda-8519bb72f171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:45.359193 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6862cc0-f915-4d13-8cda-8519bb72f171 admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:45.359464 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6862cc0-f915-4d13-8cda-8519bb72f171 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:45.359733 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6862cc0-f915-4d13-8cda-8519bb72f171 admin admin] PATH_INFO: `/v3/domains` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:45.360376 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6862cc0-f915-4d13-8cda-8519bb72f171 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 22:11:45.360706 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6862cc0-f915-4d13-8cda-8519bb72f171 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:45.362297 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6862cc0-f915-4d13-8cda-8519bb72f171 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:45.373323 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:45.376904 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1080/2160] 10.222.0.18 () {58 vars in 1153 bytes} [Sat Aug 6 22:11:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 22:11:45.381327 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabe5f68-c515-40ab-bc89-536716ef6f58 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:45.396835 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabe5f68-c515-40ab-bc89-536716ef6f58 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:45.403155 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c66c5b-732a-4108-af39-ab8628d73b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:45.403773 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c66c5b-732a-4108-af39-ab8628d73b80 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:45.404199 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c66c5b-732a-4108-af39-ab8628d73b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:45.404367 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c66c5b-732a-4108-af39-ab8628d73b80 admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:45.405034 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c66c5b-732a-4108-af39-ab8628d73b80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:45.406948 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c66c5b-732a-4108-af39-ab8628d73b80 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:45.525051 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1081/2161] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:11:45 2022] POST /identity/v3/projects => generated 377 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 22:11:45.526154 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6862cc0-f915-4d13-8cda-8519bb72f171 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:45.538038 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6862cc0-f915-4d13-8cda-8519bb72f171 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:45.608605 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac8efc7-bd01-4f23-b973-dbb410c15afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:45.609456 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac8efc7-bd01-4f23-b973-dbb410c15afd admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:45.609738 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac8efc7-bd01-4f23-b973-dbb410c15afd admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:45.610013 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac8efc7-bd01-4f23-b973-dbb410c15afd admin admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:45.610663 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac8efc7-bd01-4f23-b973-dbb410c15afd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:45.612603 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac8efc7-bd01-4f23-b973-dbb410c15afd admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:45.702234 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1081/2162] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:11:45 2022] POST /identity/v3/users => generated 289 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 22:11:45.708331 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c66c5b-732a-4108-af39-ab8628d73b80 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:45.721925 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c66c5b-732a-4108-af39-ab8628d73b80 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:45.730543 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f801a8c-6fcc-4a9d-8900-b38f2848ed98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:45.732044 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f801a8c-6fcc-4a9d-8900-b38f2848ed98 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:45.732342 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f801a8c-6fcc-4a9d-8900-b38f2848ed98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:45.732678 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f801a8c-6fcc-4a9d-8900-b38f2848ed98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:45.733984 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f801a8c-6fcc-4a9d-8900-b38f2848ed98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:45.735486 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f801a8c-6fcc-4a9d-8900-b38f2848ed98 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:45.752613 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1082/2163] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:11:45 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:11:45.757316 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac8efc7-bd01-4f23-b973-dbb410c15afd admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:45.774915 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac8efc7-bd01-4f23-b973-dbb410c15afd admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:45.780530 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a03c34f-797e-4459-a5a2-1a15a951df2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:45.781230 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a03c34f-797e-4459-a5a2-1a15a951df2f admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:45.781508 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a03c34f-797e-4459-a5a2-1a15a951df2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:45.781781 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a03c34f-797e-4459-a5a2-1a15a951df2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:45.782431 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a03c34f-797e-4459-a5a2-1a15a951df2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:45.784030 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a03c34f-797e-4459-a5a2-1a15a951df2f admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:45.796185 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1082/2164] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:11:45 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:11:45.800477 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f801a8c-6fcc-4a9d-8900-b38f2848ed98 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:45.810975 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f801a8c-6fcc-4a9d-8900-b38f2848ed98 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:45.815789 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e77a27e-0bef-4e3c-9de8-4953368af63f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:45.816429 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e77a27e-0bef-4e3c-9de8-4953368af63f admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:45.816702 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e77a27e-0bef-4e3c-9de8-4953368af63f admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:45.816975 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e77a27e-0bef-4e3c-9de8-4953368af63f admin admin] PATH_INFO: `/v3/projects/4c307a8578514139b1d2e8d51fe75fed/users/b93181a8685344a893bfec4c004589d5/roles/56e4a96094e9434cbbafb22579c034f5` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:45.833135 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e77a27e-0bef-4e3c-9de8-4953368af63f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c307a8578514139b1d2e8d51fe75fed, user_id=b93181a8685344a893bfec4c004589d5, role_id=56e4a96094e9434cbbafb22579c034f5)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:45.835931 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e77a27e-0bef-4e3c-9de8-4953368af63f admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:45.842448 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:45.932884 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1083/2165] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:11:45 2022] PUT /identity/v3/projects/4c307a8578514139b1d2e8d51fe75fed/users/b93181a8685344a893bfec4c004589d5/roles/56e4a96094e9434cbbafb22579c034f5 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 22:11:45.938331 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:45.943731 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2a3976-6330-4d2b-bfd3-a49c5aecedb4 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:45.944013 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2a3976-6330-4d2b-bfd3-a49c5aecedb4 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:45.944283 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2a3976-6330-4d2b-bfd3-a49c5aecedb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:45.975011 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-4a2a3976-6330-4d2b-bfd3-a49c5aecedb4 None None] MFA Rules not processed for user `b93181a8685344a893bfec4c004589d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:11:45.976647 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-4a2a3976-6330-4d2b-bfd3-a49c5aecedb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:46.045207 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1083/2166] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:11:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:11:46.057472 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e77a27e-0bef-4e3c-9de8-4953368af63f admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:46.059274 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-3e77a27e-0bef-4e3c-9de8-4953368af63f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:46.141133 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e77a27e-0bef-4e3c-9de8-4953368af63f admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:46.144950 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57515c1-5a94-4bba-9420-15c063f29918 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8d38e73e105476e8de858cfa939ba20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:46.145558 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57515c1-5a94-4bba-9420-15c063f29918 service glance] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:46.145846 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57515c1-5a94-4bba-9420-15c063f29918 service glance] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:46.146120 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57515c1-5a94-4bba-9420-15c063f29918 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:46.155855 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57515c1-5a94-4bba-9420-15c063f29918 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:46.157335 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57515c1-5a94-4bba-9420-15c063f29918 service glance] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:46.166478 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1084/2167] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:11:46 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 22:11:46.611590 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:46.825568 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569bbf77-531f-4e27-9381-c075482bd2e9 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:46.825697 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569bbf77-531f-4e27-9381-c075482bd2e9 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:46.826255 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569bbf77-531f-4e27-9381-c075482bd2e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:46.856214 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-569bbf77-531f-4e27-9381-c075482bd2e9 None None] MFA Rules not processed for user `56138f43f00e46ea9dda399a0fb0e822`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:11:46.857480 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-569bbf77-531f-4e27-9381-c075482bd2e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:46.926829 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1084/2168] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:11:46 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:11:46.938451 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57515c1-5a94-4bba-9420-15c063f29918 service glance] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:46.939950 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-e57515c1-5a94-4bba-9420-15c063f29918 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:46.999168 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57515c1-5a94-4bba-9420-15c063f29918 service glance] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:47.002903 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1a101c-f143-4227-83ef-fed8380a8808 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa71c71de263478da234f3c5716a6e5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:47.003511 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1a101c-f143-4227-83ef-fed8380a8808 service cinder] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:47.003969 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1a101c-f143-4227-83ef-fed8380a8808 service cinder] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:47.004254 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1a101c-f143-4227-83ef-fed8380a8808 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:47.013873 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1a101c-f143-4227-83ef-fed8380a8808 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:47.015514 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1a101c-f143-4227-83ef-fed8380a8808 service cinder] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:47.025456 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1085/2169] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:11:46 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:11:48.117207 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569bbf77-531f-4e27-9381-c075482bd2e9 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:48.119795 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-569bbf77-531f-4e27-9381-c075482bd2e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:48.176601 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:48.225896 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569bbf77-531f-4e27-9381-c075482bd2e9 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:48.248327 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c01c24c-217e-4e4a-9298-a128715b0046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:48.249194 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c01c24c-217e-4e4a-9298-a128715b0046 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:48.249376 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c01c24c-217e-4e4a-9298-a128715b0046 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:48.249719 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c01c24c-217e-4e4a-9298-a128715b0046 admin admin] PATH_INFO: `/v3/users/963687890bee4c6d95c633361caff409` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:48.256014 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c01c24c-217e-4e4a-9298-a128715b0046 admin admin] RBAC: Authorizing `identity:delete_user(user_id=963687890bee4c6d95c633361caff409)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:48.256014 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c01c24c-217e-4e4a-9298-a128715b0046 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:48.522835 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-2c01c24c-217e-4e4a-9298-a128715b0046 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '963687890bee4c6d95c633361caff409'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:48.535047 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-2c01c24c-217e-4e4a-9298-a128715b0046 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '963687890bee4c6d95c633361caff409'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:48.647873 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-2c01c24c-217e-4e4a-9298-a128715b0046 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '963687890bee4c6d95c633361caff409'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:48.656093 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-2c01c24c-217e-4e4a-9298-a128715b0046 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '963687890bee4c6d95c633361caff409'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:48.656817 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-2c01c24c-217e-4e4a-9298-a128715b0046 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '963687890bee4c6d95c633361caff409'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:48.664877 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1085/2170] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:11:48 2022] DELETE /identity/v3/users/963687890bee4c6d95c633361caff409 => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:11:48.669237 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6eeb55-839f-47b6-b217-5ef675f9aae5 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:48.669751 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6eeb55-839f-47b6-b217-5ef675f9aae5 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:48.669828 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6eeb55-839f-47b6-b217-5ef675f9aae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:48.693932 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-cf6eeb55-839f-47b6-b217-5ef675f9aae5 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:11:48.695712 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-cf6eeb55-839f-47b6-b217-5ef675f9aae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:48.787461 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1086/2171] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:11:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 22:11:48.821945 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c01c24c-217e-4e4a-9298-a128715b0046 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:48.824050 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-2c01c24c-217e-4e4a-9298-a128715b0046 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:48.912082 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c01c24c-217e-4e4a-9298-a128715b0046 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:48.930365 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d909ab7-f3cd-4b8a-a644-fb9c20ca3722 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:48.931029 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d909ab7-f3cd-4b8a-a644-fb9c20ca3722 service neutron] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:48.931291 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d909ab7-f3cd-4b8a-a644-fb9c20ca3722 service neutron] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:48.931563 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d909ab7-f3cd-4b8a-a644-fb9c20ca3722 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:48.956719 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d909ab7-f3cd-4b8a-a644-fb9c20ca3722 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:48.959139 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d909ab7-f3cd-4b8a-a644-fb9c20ca3722 service neutron] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:48.974863 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1086/2172] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:11:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:11:49.619597 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6eeb55-839f-47b6-b217-5ef675f9aae5 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:49.622026 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-cf6eeb55-839f-47b6-b217-5ef675f9aae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:49.645449 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6eeb55-839f-47b6-b217-5ef675f9aae5 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:49.664989 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f4f36f-6bc3-4b46-b70b-a8dd6e2c88c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:49.665755 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f4f36f-6bc3-4b46-b70b-a8dd6e2c88c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:49.666220 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f4f36f-6bc3-4b46-b70b-a8dd6e2c88c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:49.666595 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f4f36f-6bc3-4b46-b70b-a8dd6e2c88c6 admin admin] PATH_INFO: `/v3/projects/7fc9aab747c1420da7be0cd906dc4a7d` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:49.668765 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f4f36f-6bc3-4b46-b70b-a8dd6e2c88c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fc9aab747c1420da7be0cd906dc4a7d)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:49.670771 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f4f36f-6bc3-4b46-b70b-a8dd6e2c88c6 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:49.697727 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:49.795334 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-d6f4f36f-6bc3-4b46-b70b-a8dd6e2c88c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fc9aab747c1420da7be0cd906dc4a7d', 'request_id': 'req-d6f4f36f-6bc3-4b46-b70b-a8dd6e2c88c6'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:49.896685 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-d6f4f36f-6bc3-4b46-b70b-a8dd6e2c88c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fc9aab747c1420da7be0cd906dc4a7d', 'request_id': 'req-d6f4f36f-6bc3-4b46-b70b-a8dd6e2c88c6'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:49.903153 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-d6f4f36f-6bc3-4b46-b70b-a8dd6e2c88c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fc9aab747c1420da7be0cd906dc4a7d', 'request_id': 'req-d6f4f36f-6bc3-4b46-b70b-a8dd6e2c88c6'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:49.914482 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-d6f4f36f-6bc3-4b46-b70b-a8dd6e2c88c6 admin admin] The token cache is being invalidate because project 7fc9aab747c1420da7be0cd906dc4a7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:11:49.914773 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-d6f4f36f-6bc3-4b46-b70b-a8dd6e2c88c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:49.915532 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-d6f4f36f-6bc3-4b46-b70b-a8dd6e2c88c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:11:49.917716 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1087/2173] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:11:49 2022] DELETE /identity/v3/projects/7fc9aab747c1420da7be0cd906dc4a7d => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:11:49.923045 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c730a8-6d01-4dae-be91-34ce632d111f None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:49.923496 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c730a8-6d01-4dae-be91-34ce632d111f None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:49.923586 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c730a8-6d01-4dae-be91-34ce632d111f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:49.945093 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-f2c730a8-6d01-4dae-be91-34ce632d111f None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:11:49.946320 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-f2c730a8-6d01-4dae-be91-34ce632d111f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:49.998095 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:11:50.015468 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1087/2174] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:11:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:11:50.023293 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfada9e7-616d-4a5d-94e3-c7e68cd46039 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:50.023644 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfada9e7-616d-4a5d-94e3-c7e68cd46039 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:50.023985 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfada9e7-616d-4a5d-94e3-c7e68cd46039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:50.045475 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-cfada9e7-616d-4a5d-94e3-c7e68cd46039 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:11:50.046796 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-cfada9e7-616d-4a5d-94e3-c7e68cd46039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:50.054844 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1088/2175] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:11:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:11:50.060244 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c730a8-6d01-4dae-be91-34ce632d111f None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:50.080644 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c730a8-6d01-4dae-be91-34ce632d111f None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:50.085665 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc78cefe-b0d3-4ba4-b64e-a205c2568dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:50.086931 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc78cefe-b0d3-4ba4-b64e-a205c2568dca admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:50.086931 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc78cefe-b0d3-4ba4-b64e-a205c2568dca admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:50.086931 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc78cefe-b0d3-4ba4-b64e-a205c2568dca admin admin] PATH_INFO: `/v3/domains` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:50.087466 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc78cefe-b0d3-4ba4-b64e-a205c2568dca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 22:11:50.087763 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc78cefe-b0d3-4ba4-b64e-a205c2568dca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:50.090297 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc78cefe-b0d3-4ba4-b64e-a205c2568dca admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:50.107869 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1088/2176] 10.222.0.18 () {58 vars in 1153 bytes} [Sat Aug 6 22:11:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 22:11:50.113237 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfada9e7-616d-4a5d-94e3-c7e68cd46039 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:50.124776 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfada9e7-616d-4a5d-94e3-c7e68cd46039 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:50.131326 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777fe1b0-77fc-46aa-8d9a-958aa1568f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:50.132686 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777fe1b0-77fc-46aa-8d9a-958aa1568f64 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:50.132686 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777fe1b0-77fc-46aa-8d9a-958aa1568f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:50.133102 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777fe1b0-77fc-46aa-8d9a-958aa1568f64 admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:50.133865 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777fe1b0-77fc-46aa-8d9a-958aa1568f64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:50.136382 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777fe1b0-77fc-46aa-8d9a-958aa1568f64 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:50.260675 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1089/2177] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:11:50 2022] POST /identity/v3/projects => generated 375 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 22:11:50.264944 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc78cefe-b0d3-4ba4-b64e-a205c2568dca admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:50.275166 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc78cefe-b0d3-4ba4-b64e-a205c2568dca admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:50.344177 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a172aa-369c-42bb-b50b-4fb07b8b2313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:50.344818 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a172aa-369c-42bb-b50b-4fb07b8b2313 admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:50.345096 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a172aa-369c-42bb-b50b-4fb07b8b2313 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:50.345365 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a172aa-369c-42bb-b50b-4fb07b8b2313 admin admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:50.346012 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a172aa-369c-42bb-b50b-4fb07b8b2313 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:50.348682 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a172aa-369c-42bb-b50b-4fb07b8b2313 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:50.390719 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204d5892-421a-427b-83a2-1746576cc6c6 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:50.391001 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204d5892-421a-427b-83a2-1746576cc6c6 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:50.391271 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204d5892-421a-427b-83a2-1746576cc6c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:50.410136 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-204d5892-421a-427b-83a2-1746576cc6c6 None None] MFA Rules not processed for user `56138f43f00e46ea9dda399a0fb0e822`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:11:50.411321 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-204d5892-421a-427b-83a2-1746576cc6c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:50.463737 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1089/2178] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:11:50 2022] POST /identity/v3/users => generated 288 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:11:50.468334 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a172aa-369c-42bb-b50b-4fb07b8b2313 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:50.472456 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1090/2179] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:11:50 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 22:11:50.479232 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a172aa-369c-42bb-b50b-4fb07b8b2313 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:50.484791 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36355e09-87b2-47f8-9cc0-17023451a1fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:50.485404 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36355e09-87b2-47f8-9cc0-17023451a1fe admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:50.485686 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36355e09-87b2-47f8-9cc0-17023451a1fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:50.485961 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36355e09-87b2-47f8-9cc0-17023451a1fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:50.486630 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36355e09-87b2-47f8-9cc0-17023451a1fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:50.488229 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36355e09-87b2-47f8-9cc0-17023451a1fe admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:50.500696 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1090/2180] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:11:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 22:11:50.504549 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204d5892-421a-427b-83a2-1746576cc6c6 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:50.516937 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204d5892-421a-427b-83a2-1746576cc6c6 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:50.523069 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6518c4-0cdd-4494-a22d-0d568099ee8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:50.523757 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6518c4-0cdd-4494-a22d-0d568099ee8b admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:50.524223 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6518c4-0cdd-4494-a22d-0d568099ee8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:50.524592 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6518c4-0cdd-4494-a22d-0d568099ee8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:50.525378 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6518c4-0cdd-4494-a22d-0d568099ee8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:50.527792 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6518c4-0cdd-4494-a22d-0d568099ee8b admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:50.545368 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1091/2181] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:11:50 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:11:50.546477 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36355e09-87b2-47f8-9cc0-17023451a1fe admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:50.556845 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36355e09-87b2-47f8-9cc0-17023451a1fe admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:50.561966 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bf9b2d-6023-4dfc-9552-3eec597f4bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:50.562655 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf9b2d-6023-4dfc-9552-3eec597f4bf3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:50.562934 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf9b2d-6023-4dfc-9552-3eec597f4bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:50.563257 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf9b2d-6023-4dfc-9552-3eec597f4bf3 admin admin] PATH_INFO: `/v3/projects/af8c42268adf44be9b5c54e7feb09a51/users/aba1a15fd81747e8bf792fad56857023/roles/56e4a96094e9434cbbafb22579c034f5` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:50.579262 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bf9b2d-6023-4dfc-9552-3eec597f4bf3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af8c42268adf44be9b5c54e7feb09a51, user_id=aba1a15fd81747e8bf792fad56857023, role_id=56e4a96094e9434cbbafb22579c034f5)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:50.582016 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bf9b2d-6023-4dfc-9552-3eec597f4bf3 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:50.697622 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1091/2182] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:11:50 2022] PUT /identity/v3/projects/af8c42268adf44be9b5c54e7feb09a51/users/aba1a15fd81747e8bf792fad56857023/roles/56e4a96094e9434cbbafb22579c034f5 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:11:50.709296 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d25512-049d-4516-ac73-944ef014e819 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:50.709843 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d25512-049d-4516-ac73-944ef014e819 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:50.709843 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d25512-049d-4516-ac73-944ef014e819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:50.739782 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-a1d25512-049d-4516-ac73-944ef014e819 None None] MFA Rules not processed for user `aba1a15fd81747e8bf792fad56857023`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:11:50.741051 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-a1d25512-049d-4516-ac73-944ef014e819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:50.809915 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1092/2183] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:11:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:11:50.821710 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bf9b2d-6023-4dfc-9552-3eec597f4bf3 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:11:50.824372 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-18bf9b2d-6023-4dfc-9552-3eec597f4bf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:11:50.886803 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bf9b2d-6023-4dfc-9552-3eec597f4bf3 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:11:50.890769 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1e053b-e828-4f8e-9bc9-06144a6ffa44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa71c71de263478da234f3c5716a6e5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:11:50.893100 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e053b-e828-4f8e-9bc9-06144a6ffa44 service cinder] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:11:50.893100 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e053b-e828-4f8e-9bc9-06144a6ffa44 service cinder] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:11:50.893100 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e053b-e828-4f8e-9bc9-06144a6ffa44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:11:50.900676 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1e053b-e828-4f8e-9bc9-06144a6ffa44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:11:50.902819 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1e053b-e828-4f8e-9bc9-06144a6ffa44 service cinder] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:11:50.911943 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1092/2184] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:11:50 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:12:00.806037 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:12:00.907615 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=80,keystone:INSERT=4 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:12:15.158887 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d25512-049d-4516-ac73-944ef014e819 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:15.227374 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d25512-049d-4516-ac73-944ef014e819 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:15.232954 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9604b781-139e-4b06-92a8-0af8d34774aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:15.233583 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9604b781-139e-4b06-92a8-0af8d34774aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:15.233866 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9604b781-139e-4b06-92a8-0af8d34774aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:15.234138 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9604b781-139e-4b06-92a8-0af8d34774aa admin admin] PATH_INFO: `/v3/users/aba1a15fd81747e8bf792fad56857023` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:15.235279 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9604b781-139e-4b06-92a8-0af8d34774aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=aba1a15fd81747e8bf792fad56857023)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:15.237301 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9604b781-139e-4b06-92a8-0af8d34774aa admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:15.509989 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-9604b781-139e-4b06-92a8-0af8d34774aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aba1a15fd81747e8bf792fad56857023'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:15.521639 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-9604b781-139e-4b06-92a8-0af8d34774aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aba1a15fd81747e8bf792fad56857023'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:15.528837 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-9604b781-139e-4b06-92a8-0af8d34774aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aba1a15fd81747e8bf792fad56857023'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:15.529495 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-9604b781-139e-4b06-92a8-0af8d34774aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aba1a15fd81747e8bf792fad56857023'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:15.530102 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-9604b781-139e-4b06-92a8-0af8d34774aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aba1a15fd81747e8bf792fad56857023'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:15.656092 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1093/2185] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:12:15 2022] DELETE /identity/v3/users/aba1a15fd81747e8bf792fad56857023 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 22:12:15.660559 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ed000d-b57d-48c3-8d15-4e0dd96366ae None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:15.661071 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ed000d-b57d-48c3-8d15-4e0dd96366ae None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:15.661163 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ed000d-b57d-48c3-8d15-4e0dd96366ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:15.681364 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-c0ed000d-b57d-48c3-8d15-4e0dd96366ae None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:12:15.684746 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-c0ed000d-b57d-48c3-8d15-4e0dd96366ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:12:15.760498 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1093/2186] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:12:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:12:15.771604 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9604b781-139e-4b06-92a8-0af8d34774aa admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:15.773538 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-9604b781-139e-4b06-92a8-0af8d34774aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:12:15.841720 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9604b781-139e-4b06-92a8-0af8d34774aa admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:15.845743 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565e6333-bcf3-468d-8f52-c4dde20c45ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:15.846492 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565e6333-bcf3-468d-8f52-c4dde20c45ce service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:15.846832 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565e6333-bcf3-468d-8f52-c4dde20c45ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:15.847166 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565e6333-bcf3-468d-8f52-c4dde20c45ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:15.864072 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565e6333-bcf3-468d-8f52-c4dde20c45ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:15.865571 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565e6333-bcf3-468d-8f52-c4dde20c45ce service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:15.875913 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1094/2187] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:12:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:12:16.377406 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ed000d-b57d-48c3-8d15-4e0dd96366ae None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:16.379285 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-c0ed000d-b57d-48c3-8d15-4e0dd96366ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:12:16.391763 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ed000d-b57d-48c3-8d15-4e0dd96366ae None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:16.396429 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5424fcc-b8ec-4560-88bc-f53ad711d8e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:16.396942 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5424fcc-b8ec-4560-88bc-f53ad711d8e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:16.397220 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5424fcc-b8ec-4560-88bc-f53ad711d8e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:16.397492 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5424fcc-b8ec-4560-88bc-f53ad711d8e1 admin admin] PATH_INFO: `/v3/projects/af8c42268adf44be9b5c54e7feb09a51` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:16.398615 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5424fcc-b8ec-4560-88bc-f53ad711d8e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af8c42268adf44be9b5c54e7feb09a51)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:16.400821 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5424fcc-b8ec-4560-88bc-f53ad711d8e1 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:16.554270 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-f5424fcc-b8ec-4560-88bc-f53ad711d8e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af8c42268adf44be9b5c54e7feb09a51', 'request_id': 'req-f5424fcc-b8ec-4560-88bc-f53ad711d8e1'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:16.650228 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-f5424fcc-b8ec-4560-88bc-f53ad711d8e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af8c42268adf44be9b5c54e7feb09a51', 'request_id': 'req-f5424fcc-b8ec-4560-88bc-f53ad711d8e1'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:16.656590 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-f5424fcc-b8ec-4560-88bc-f53ad711d8e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af8c42268adf44be9b5c54e7feb09a51', 'request_id': 'req-f5424fcc-b8ec-4560-88bc-f53ad711d8e1'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:16.668363 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-f5424fcc-b8ec-4560-88bc-f53ad711d8e1 admin admin] The token cache is being invalidate because project af8c42268adf44be9b5c54e7feb09a51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:12:16.668980 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-f5424fcc-b8ec-4560-88bc-f53ad711d8e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:16.669307 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-f5424fcc-b8ec-4560-88bc-f53ad711d8e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:16.670914 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1094/2188] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:12:16 2022] DELETE /identity/v3/projects/af8c42268adf44be9b5c54e7feb09a51 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:12:16.676116 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfe34d3-e36d-4b3f-8291-507fcd1fdb10 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:16.676627 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfe34d3-e36d-4b3f-8291-507fcd1fdb10 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:16.676739 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfe34d3-e36d-4b3f-8291-507fcd1fdb10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:16.696800 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-1cfe34d3-e36d-4b3f-8291-507fcd1fdb10 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:12:16.698639 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-1cfe34d3-e36d-4b3f-8291-507fcd1fdb10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:12:16.715775 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:12:16.759324 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1095/2189] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:12:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 22:12:16.768667 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94851bd3-913f-4d4f-b9d9-31df124ee0ac None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:16.768945 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94851bd3-913f-4d4f-b9d9-31df124ee0ac None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:16.769290 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94851bd3-913f-4d4f-b9d9-31df124ee0ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:16.790080 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-94851bd3-913f-4d4f-b9d9-31df124ee0ac None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:12:16.791709 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-94851bd3-913f-4d4f-b9d9-31df124ee0ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:12:16.800441 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1095/2190] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:12:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:12:16.804776 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfe34d3-e36d-4b3f-8291-507fcd1fdb10 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:16.815039 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfe34d3-e36d-4b3f-8291-507fcd1fdb10 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:16.819811 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dc6001-def2-439d-932c-88751071d26a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:16.820453 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dc6001-def2-439d-932c-88751071d26a admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:16.820728 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dc6001-def2-439d-932c-88751071d26a admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:16.820999 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dc6001-def2-439d-932c-88751071d26a admin admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:16.825154 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dc6001-def2-439d-932c-88751071d26a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 22:12:16.825154 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dc6001-def2-439d-932c-88751071d26a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:16.825154 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dc6001-def2-439d-932c-88751071d26a admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:16.837280 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1096/2191] 10.222.0.18 () {58 vars in 1153 bytes} [Sat Aug 6 22:12:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 22:12:16.842843 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94851bd3-913f-4d4f-b9d9-31df124ee0ac None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:16.853266 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94851bd3-913f-4d4f-b9d9-31df124ee0ac None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:16.858567 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caaf9e1-37c5-483b-ab40-bed491b928aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:16.859222 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caaf9e1-37c5-483b-ab40-bed491b928aa admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:16.859660 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caaf9e1-37c5-483b-ab40-bed491b928aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:16.860070 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caaf9e1-37c5-483b-ab40-bed491b928aa admin admin] PATH_INFO: `/v3/projects` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:16.861356 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caaf9e1-37c5-483b-ab40-bed491b928aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:16.863598 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caaf9e1-37c5-483b-ab40-bed491b928aa admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:17.026518 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1096/2192] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:12:16 2022] POST /identity/v3/projects => generated 373 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:12:17.030938 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dc6001-def2-439d-932c-88751071d26a admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:17.041884 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dc6001-def2-439d-932c-88751071d26a admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:17.104360 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabc3fbc-32fc-4f32-912a-7c15ba0f0648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:17.107856 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabc3fbc-32fc-4f32-912a-7c15ba0f0648 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:17.107856 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabc3fbc-32fc-4f32-912a-7c15ba0f0648 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:17.107856 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabc3fbc-32fc-4f32-912a-7c15ba0f0648 admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:17.107856 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabc3fbc-32fc-4f32-912a-7c15ba0f0648 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:17.108136 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabc3fbc-32fc-4f32-912a-7c15ba0f0648 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:17.217563 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1097/2193] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:12:17 2022] POST /identity/v3/users => generated 287 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:12:17.222503 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caaf9e1-37c5-483b-ab40-bed491b928aa admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:17.235947 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caaf9e1-37c5-483b-ab40-bed491b928aa admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:17.241382 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2f388f-a5a2-492c-a7e3-2453b053a4cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:17.241987 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2f388f-a5a2-492c-a7e3-2453b053a4cb admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:17.242262 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2f388f-a5a2-492c-a7e3-2453b053a4cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:17.242627 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2f388f-a5a2-492c-a7e3-2453b053a4cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:17.243278 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2f388f-a5a2-492c-a7e3-2453b053a4cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:17.245318 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2f388f-a5a2-492c-a7e3-2453b053a4cb admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:17.256329 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1097/2194] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:12:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:12:17.270493 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabc3fbc-32fc-4f32-912a-7c15ba0f0648 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:17.281479 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabc3fbc-32fc-4f32-912a-7c15ba0f0648 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:17.286304 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad051e7-f2b1-4740-8d37-c30d56e1a2ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:17.286894 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad051e7-f2b1-4740-8d37-c30d56e1a2ad admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:17.287168 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad051e7-f2b1-4740-8d37-c30d56e1a2ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:17.287436 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad051e7-f2b1-4740-8d37-c30d56e1a2ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:17.288071 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad051e7-f2b1-4740-8d37-c30d56e1a2ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:17.289879 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad051e7-f2b1-4740-8d37-c30d56e1a2ad admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:17.305876 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1098/2195] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:12:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:12:17.307005 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2f388f-a5a2-492c-a7e3-2453b053a4cb admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:17.319476 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2f388f-a5a2-492c-a7e3-2453b053a4cb admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:17.326713 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1c9181-877c-42f1-af5f-33c844d17174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:17.328787 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1c9181-877c-42f1-af5f-33c844d17174 admin admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:17.332641 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1c9181-877c-42f1-af5f-33c844d17174 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:17.332641 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1c9181-877c-42f1-af5f-33c844d17174 admin admin] PATH_INFO: `/v3/projects/6354597f8edf4ebc9539a33314e6bc06/users/9469cd012c2549288ba403902b667295/roles/56e4a96094e9434cbbafb22579c034f5` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:17.357480 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1c9181-877c-42f1-af5f-33c844d17174 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6354597f8edf4ebc9539a33314e6bc06, user_id=9469cd012c2549288ba403902b667295, role_id=56e4a96094e9434cbbafb22579c034f5)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:17.360330 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1c9181-877c-42f1-af5f-33c844d17174 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:17.473652 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1098/2196] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:12:17 2022] PUT /identity/v3/projects/6354597f8edf4ebc9539a33314e6bc06/users/9469cd012c2549288ba403902b667295/roles/56e4a96094e9434cbbafb22579c034f5 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:12:17.479684 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b423392-2eba-4a9b-b9cd-e07415df0703 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:17.480024 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b423392-2eba-4a9b-b9cd-e07415df0703 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:17.480371 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b423392-2eba-4a9b-b9cd-e07415df0703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:17.501391 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-7b423392-2eba-4a9b-b9cd-e07415df0703 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:12:17.502647 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-7b423392-2eba-4a9b-b9cd-e07415df0703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:12:17.563387 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1099/2197] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:12:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:12:17.573994 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1c9181-877c-42f1-af5f-33c844d17174 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:17.575709 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-7f1c9181-877c-42f1-af5f-33c844d17174 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:12:17.598264 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:12:17.636076 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1c9181-877c-42f1-af5f-33c844d17174 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:17.639808 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fbf8b3-4d66-48db-897b-e7a4db1aca60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:17.640438 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fbf8b3-4d66-48db-897b-e7a4db1aca60 service neutron] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:17.640744 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fbf8b3-4d66-48db-897b-e7a4db1aca60 service neutron] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:17.641012 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fbf8b3-4d66-48db-897b-e7a4db1aca60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:17.652830 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fbf8b3-4d66-48db-897b-e7a4db1aca60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:17.656305 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fbf8b3-4d66-48db-897b-e7a4db1aca60 service neutron] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:17.672958 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1099/2198] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:12:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:12:17.758570 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b423392-2eba-4a9b-b9cd-e07415df0703 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:17.760080 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-7b423392-2eba-4a9b-b9cd-e07415df0703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:12:17.771835 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b423392-2eba-4a9b-b9cd-e07415df0703 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:17.776522 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186f6ce0-8b31-456e-a95a-730f6973a7ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:17.777351 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f6ce0-8b31-456e-a95a-730f6973a7ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:17.777632 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f6ce0-8b31-456e-a95a-730f6973a7ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:17.777899 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f6ce0-8b31-456e-a95a-730f6973a7ad admin admin] PATH_INFO: `/v3/users/1f36a772b21e4114b160d2b51e7e6825` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:17.779019 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186f6ce0-8b31-456e-a95a-730f6973a7ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f36a772b21e4114b160d2b51e7e6825)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:17.782881 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186f6ce0-8b31-456e-a95a-730f6973a7ad admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:17.806152 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:12:18.046602 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-186f6ce0-8b31-456e-a95a-730f6973a7ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f36a772b21e4114b160d2b51e7e6825'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:18.058614 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-186f6ce0-8b31-456e-a95a-730f6973a7ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f36a772b21e4114b160d2b51e7e6825'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:18.071768 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-186f6ce0-8b31-456e-a95a-730f6973a7ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f36a772b21e4114b160d2b51e7e6825'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:18.072715 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-186f6ce0-8b31-456e-a95a-730f6973a7ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f36a772b21e4114b160d2b51e7e6825'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:18.073655 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-186f6ce0-8b31-456e-a95a-730f6973a7ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f36a772b21e4114b160d2b51e7e6825'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:18.170842 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1100/2199] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:12:17 2022] DELETE /identity/v3/users/1f36a772b21e4114b160d2b51e7e6825 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:12:18.175299 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266ace45-943d-4a07-b9c4-7c9f4057b722 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:18.175650 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266ace45-943d-4a07-b9c4-7c9f4057b722 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:18.176018 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266ace45-943d-4a07-b9c4-7c9f4057b722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:18.195063 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-266ace45-943d-4a07-b9c4-7c9f4057b722 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:12:18.196253 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-266ace45-943d-4a07-b9c4-7c9f4057b722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:12:18.254664 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1100/2200] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:12:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:12:18.265040 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186f6ce0-8b31-456e-a95a-730f6973a7ad admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:18.266466 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-186f6ce0-8b31-456e-a95a-730f6973a7ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:12:18.330598 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186f6ce0-8b31-456e-a95a-730f6973a7ad admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:18.334275 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2cd7b9-ba68-4288-bb56-71f2ddb90447 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:18.334954 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2cd7b9-ba68-4288-bb56-71f2ddb90447 service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:18.335301 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2cd7b9-ba68-4288-bb56-71f2ddb90447 service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:18.335577 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2cd7b9-ba68-4288-bb56-71f2ddb90447 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:18.345956 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2cd7b9-ba68-4288-bb56-71f2ddb90447 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:18.347385 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2cd7b9-ba68-4288-bb56-71f2ddb90447 service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:18.359218 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1101/2201] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:12:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:12:18.994042 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266ace45-943d-4a07-b9c4-7c9f4057b722 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:18.996049 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-266ace45-943d-4a07-b9c4-7c9f4057b722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:12:19.011578 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266ace45-943d-4a07-b9c4-7c9f4057b722 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:19.017142 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b994f6-5e3b-4d53-bafb-bfb8be048040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:19.017920 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b994f6-5e3b-4d53-bafb-bfb8be048040 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:19.018298 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b994f6-5e3b-4d53-bafb-bfb8be048040 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:19.018762 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b994f6-5e3b-4d53-bafb-bfb8be048040 admin admin] PATH_INFO: `/v3/projects/0a6a5feaec104d728c2d182a36a683c5` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:19.020610 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b994f6-5e3b-4d53-bafb-bfb8be048040 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a6a5feaec104d728c2d182a36a683c5)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:19.022652 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b994f6-5e3b-4d53-bafb-bfb8be048040 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:19.178235 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-b1b994f6-5e3b-4d53-bafb-bfb8be048040 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a6a5feaec104d728c2d182a36a683c5', 'request_id': 'req-b1b994f6-5e3b-4d53-bafb-bfb8be048040'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:19.278861 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-b1b994f6-5e3b-4d53-bafb-bfb8be048040 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a6a5feaec104d728c2d182a36a683c5', 'request_id': 'req-b1b994f6-5e3b-4d53-bafb-bfb8be048040'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:19.286232 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-b1b994f6-5e3b-4d53-bafb-bfb8be048040 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a6a5feaec104d728c2d182a36a683c5', 'request_id': 'req-b1b994f6-5e3b-4d53-bafb-bfb8be048040'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:19.300232 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-b1b994f6-5e3b-4d53-bafb-bfb8be048040 admin admin] The token cache is being invalidate because project 0a6a5feaec104d728c2d182a36a683c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:12:19.300682 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-b1b994f6-5e3b-4d53-bafb-bfb8be048040 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:19.301422 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-b1b994f6-5e3b-4d53-bafb-bfb8be048040 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:19.302994 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1101/2202] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:12:18 2022] DELETE /identity/v3/projects/0a6a5feaec104d728c2d182a36a683c5 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:12:19.308566 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a0cf4e-d015-497e-9521-b77c44fda445 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:19.309038 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a0cf4e-d015-497e-9521-b77c44fda445 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:19.309137 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a0cf4e-d015-497e-9521-b77c44fda445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:19.329269 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-d3a0cf4e-d015-497e-9521-b77c44fda445 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:12:19.330482 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-d3a0cf4e-d015-497e-9521-b77c44fda445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:12:19.407367 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1102/2203] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:12:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:12:19.409524 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d7d5eb-8380-436b-99cf-d8672966fe7c None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:19.409812 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d7d5eb-8380-436b-99cf-d8672966fe7c None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:19.410089 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d7d5eb-8380-436b-99cf-d8672966fe7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:19.434091 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-41d7d5eb-8380-436b-99cf-d8672966fe7c None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:12:19.435442 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-41d7d5eb-8380-436b-99cf-d8672966fe7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:12:19.444591 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1102/2204] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:12:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:12:19.449025 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a0cf4e-d015-497e-9521-b77c44fda445 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:19.461973 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a0cf4e-d015-497e-9521-b77c44fda445 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:19.468117 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf05b7aa-8abb-445f-b8a1-adabd0ce9ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:19.468796 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf05b7aa-8abb-445f-b8a1-adabd0ce9ca9 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:19.469078 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf05b7aa-8abb-445f-b8a1-adabd0ce9ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:19.469355 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf05b7aa-8abb-445f-b8a1-adabd0ce9ca9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:19.470011 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf05b7aa-8abb-445f-b8a1-adabd0ce9ca9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 22:12:19.470316 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf05b7aa-8abb-445f-b8a1-adabd0ce9ca9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:19.471938 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf05b7aa-8abb-445f-b8a1-adabd0ce9ca9 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:19.486273 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1103/2205] 10.222.0.18 () {58 vars in 1153 bytes} [Sat Aug 6 22:12:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 22:12:19.490281 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d7d5eb-8380-436b-99cf-d8672966fe7c None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:19.500211 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d7d5eb-8380-436b-99cf-d8672966fe7c None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:19.505707 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e731b24d-ecdc-4970-8f7e-0247b7ef59a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:19.506457 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731b24d-ecdc-4970-8f7e-0247b7ef59a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:19.506848 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731b24d-ecdc-4970-8f7e-0247b7ef59a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:19.507557 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731b24d-ecdc-4970-8f7e-0247b7ef59a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:19.507965 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e731b24d-ecdc-4970-8f7e-0247b7ef59a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:19.510400 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e731b24d-ecdc-4970-8f7e-0247b7ef59a4 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:19.655413 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1103/2206] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:12:19 2022] POST /identity/v3/projects => generated 375 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:12:19.656714 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf05b7aa-8abb-445f-b8a1-adabd0ce9ca9 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:19.667620 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf05b7aa-8abb-445f-b8a1-adabd0ce9ca9 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:19.698112 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:12:19.726425 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d8338c-5c4d-4668-a2cc-008177bd37ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:19.727029 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d8338c-5c4d-4668-a2cc-008177bd37ed admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:19.727306 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d8338c-5c4d-4668-a2cc-008177bd37ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:19.727576 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d8338c-5c4d-4668-a2cc-008177bd37ed admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:19.728197 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d8338c-5c4d-4668-a2cc-008177bd37ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:19.730856 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d8338c-5c4d-4668-a2cc-008177bd37ed admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:19.866477 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1104/2207] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:12:19 2022] POST /identity/v3/users => generated 288 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 22:12:19.867899 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e731b24d-ecdc-4970-8f7e-0247b7ef59a4 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:19.890836 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e731b24d-ecdc-4970-8f7e-0247b7ef59a4 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:19.899035 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b41559d-9fc4-4b76-8800-bc638815038c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:19.901219 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b41559d-9fc4-4b76-8800-bc638815038c admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:19.901987 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b41559d-9fc4-4b76-8800-bc638815038c admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:19.902466 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b41559d-9fc4-4b76-8800-bc638815038c admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:19.903291 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b41559d-9fc4-4b76-8800-bc638815038c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:19.905412 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b41559d-9fc4-4b76-8800-bc638815038c admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:19.939244 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1104/2208] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:12:19 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:12:19.940285 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d8338c-5c4d-4668-a2cc-008177bd37ed admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:19.964687 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d8338c-5c4d-4668-a2cc-008177bd37ed admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:19.972368 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9222daff-989a-4fd7-afb4-ab01bc58b743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:19.974039 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9222daff-989a-4fd7-afb4-ab01bc58b743 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:19.974897 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9222daff-989a-4fd7-afb4-ab01bc58b743 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:19.976140 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9222daff-989a-4fd7-afb4-ab01bc58b743 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:19.977022 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9222daff-989a-4fd7-afb4-ab01bc58b743 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:19.981674 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9222daff-989a-4fd7-afb4-ab01bc58b743 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:20.021171 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1105/2209] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:12:19 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 22:12:20.023767 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b41559d-9fc4-4b76-8800-bc638815038c admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:20.040158 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b41559d-9fc4-4b76-8800-bc638815038c admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:20.053050 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a8d695-f5c1-4569-aa1c-699fa82dc1cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:20.055112 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a8d695-f5c1-4569-aa1c-699fa82dc1cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:20.055416 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a8d695-f5c1-4569-aa1c-699fa82dc1cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:20.056493 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a8d695-f5c1-4569-aa1c-699fa82dc1cd admin admin] PATH_INFO: `/v3/projects/6b6e295408f442b3b6c4b4da326531b4/users/beef1fabdda74b3680ae544f3011e7e4/roles/56e4a96094e9434cbbafb22579c034f5` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:20.060960 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:12:20.096009 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a8d695-f5c1-4569-aa1c-699fa82dc1cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b6e295408f442b3b6c4b4da326531b4, user_id=beef1fabdda74b3680ae544f3011e7e4, role_id=56e4a96094e9434cbbafb22579c034f5)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:20.100056 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a8d695-f5c1-4569-aa1c-699fa82dc1cd admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:20.198838 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1105/2210] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:12:20 2022] PUT /identity/v3/projects/6b6e295408f442b3b6c4b4da326531b4/users/beef1fabdda74b3680ae544f3011e7e4/roles/56e4a96094e9434cbbafb22579c034f5 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:12:20.210479 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb62d915-309a-4f53-964e-a33aeea9493b None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:20.210767 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb62d915-309a-4f53-964e-a33aeea9493b None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:20.211046 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb62d915-309a-4f53-964e-a33aeea9493b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:20.257597 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-bb62d915-309a-4f53-964e-a33aeea9493b None None] MFA Rules not processed for user `beef1fabdda74b3680ae544f3011e7e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:12:20.259234 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-bb62d915-309a-4f53-964e-a33aeea9493b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:12:20.418558 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1106/2211] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:12:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:12:20.427750 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a8d695-f5c1-4569-aa1c-699fa82dc1cd admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:20.429669 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-80a8d695-f5c1-4569-aa1c-699fa82dc1cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:12:20.530689 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a8d695-f5c1-4569-aa1c-699fa82dc1cd admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:20.535107 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29826988-076a-4621-9be1-c23afbd8f070 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa71c71de263478da234f3c5716a6e5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:20.535729 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29826988-076a-4621-9be1-c23afbd8f070 service cinder] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:20.536008 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29826988-076a-4621-9be1-c23afbd8f070 service cinder] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:20.538336 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29826988-076a-4621-9be1-c23afbd8f070 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:20.566837 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29826988-076a-4621-9be1-c23afbd8f070 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:20.579754 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29826988-076a-4621-9be1-c23afbd8f070 service cinder] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:20.595851 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1106/2212] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:12:20 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 22:12:23.986521 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127f2fd9-fab7-4ec8-9f47-1bd49d029f7c None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:23.987018 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127f2fd9-fab7-4ec8-9f47-1bd49d029f7c None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:23.987392 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127f2fd9-fab7-4ec8-9f47-1bd49d029f7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:24.008640 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-127f2fd9-fab7-4ec8-9f47-1bd49d029f7c None None] MFA Rules not processed for user `26b6e4a0640548dd8d6bb800d1f26b95`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:12:24.010054 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-127f2fd9-fab7-4ec8-9f47-1bd49d029f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:12:24.124686 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1107/2213] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:12:23 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:12:24.302361 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29826988-076a-4621-9be1-c23afbd8f070 service cinder] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:24.304140 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-29826988-076a-4621-9be1-c23afbd8f070 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:12:24.484254 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29826988-076a-4621-9be1-c23afbd8f070 service cinder] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:24.494606 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96299ad-29f1-492d-a240-aeb283dd38be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:24.495629 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96299ad-29f1-492d-a240-aeb283dd38be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:24.496117 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96299ad-29f1-492d-a240-aeb283dd38be admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:24.496642 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96299ad-29f1-492d-a240-aeb283dd38be admin admin] PATH_INFO: `/v3/users/b93181a8685344a893bfec4c004589d5` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:24.498777 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96299ad-29f1-492d-a240-aeb283dd38be admin admin] RBAC: Authorizing `identity:delete_user(user_id=b93181a8685344a893bfec4c004589d5)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:24.501104 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96299ad-29f1-492d-a240-aeb283dd38be admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:24.844710 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ed70e-80f2-40d6-89ef-ad76963160f6 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:24.845375 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ed70e-80f2-40d6-89ef-ad76963160f6 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:24.845844 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ed70e-80f2-40d6-89ef-ad76963160f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:24.872715 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-d96299ad-29f1-492d-a240-aeb283dd38be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b93181a8685344a893bfec4c004589d5'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:24.894081 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-4b7ed70e-80f2-40d6-89ef-ad76963160f6 None None] MFA Rules not processed for user `9469cd012c2549288ba403902b667295`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:12:24.895814 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-4b7ed70e-80f2-40d6-89ef-ad76963160f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:12:24.897258 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-d96299ad-29f1-492d-a240-aeb283dd38be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b93181a8685344a893bfec4c004589d5'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:24.962126 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:12:25.020987 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1108/2214] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:12:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:12:25.025832 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7ed70e-80f2-40d6-89ef-ad76963160f6 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:25.027872 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-4b7ed70e-80f2-40d6-89ef-ad76963160f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:12:25.038614 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-d96299ad-29f1-492d-a240-aeb283dd38be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b93181a8685344a893bfec4c004589d5'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:25.044709 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7ed70e-80f2-40d6-89ef-ad76963160f6 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:25.055328 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0d7f85-848f-4f72-ba2d-60e3fd152fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:25.055328 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0d7f85-848f-4f72-ba2d-60e3fd152fb5 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:25.055328 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0d7f85-848f-4f72-ba2d-60e3fd152fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:25.055328 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0d7f85-848f-4f72-ba2d-60e3fd152fb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:25.056057 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-d96299ad-29f1-492d-a240-aeb283dd38be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b93181a8685344a893bfec4c004589d5'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:25.056160 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0d7f85-848f-4f72-ba2d-60e3fd152fb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:25.056247 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-d96299ad-29f1-492d-a240-aeb283dd38be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b93181a8685344a893bfec4c004589d5'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:25.056331 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0d7f85-848f-4f72-ba2d-60e3fd152fb5 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:25.064748 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1107/2215] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:12:24 2022] DELETE /identity/v3/users/b93181a8685344a893bfec4c004589d5 => generated 0 bytes in 763 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:12:25.070296 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b877e5-62ec-4ae9-828d-8378d8e67112 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:25.070650 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b877e5-62ec-4ae9-828d-8378d8e67112 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:25.070992 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b877e5-62ec-4ae9-828d-8378d8e67112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:25.102006 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:12:25.147486 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-59b877e5-62ec-4ae9-828d-8378d8e67112 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:12:25.148991 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-59b877e5-62ec-4ae9-828d-8378d8e67112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:12:25.180516 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1109/2216] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:12:25 2022] POST /identity/v3/projects => generated 371 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 22:12:25.187533 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0d7f85-848f-4f72-ba2d-60e3fd152fb5 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:25.191244 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-1f0d7f85-848f-4f72-ba2d-60e3fd152fb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:12:25.331577 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1108/2217] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:12:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:12:25.342600 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b877e5-62ec-4ae9-828d-8378d8e67112 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:25.344155 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-59b877e5-62ec-4ae9-828d-8378d8e67112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:12:25.365578 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0d7f85-848f-4f72-ba2d-60e3fd152fb5 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:25.372225 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbb3506-265a-4465-a117-18174bb48c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:25.373007 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbb3506-265a-4465-a117-18174bb48c7d admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:25.373619 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbb3506-265a-4465-a117-18174bb48c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:25.374255 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbb3506-265a-4465-a117-18174bb48c7d admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:25.374939 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbb3506-265a-4465-a117-18174bb48c7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:25.377392 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbb3506-265a-4465-a117-18174bb48c7d admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:25.467501 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b877e5-62ec-4ae9-828d-8378d8e67112 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:25.560265 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1110/2218] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:12:25 2022] POST /identity/v3/users => generated 292 bytes in 375 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:12:25.564868 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbb3506-265a-4465-a117-18174bb48c7d admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:25.587212 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affa8b1f-bd71-4185-bb23-9ddea9343281 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:25.587212 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affa8b1f-bd71-4185-bb23-9ddea9343281 service neutron] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:25.587212 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affa8b1f-bd71-4185-bb23-9ddea9343281 service neutron] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:25.587212 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affa8b1f-bd71-4185-bb23-9ddea9343281 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:25.609825 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affa8b1f-bd71-4185-bb23-9ddea9343281 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:25.611286 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affa8b1f-bd71-4185-bb23-9ddea9343281 service neutron] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:25.612412 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbb3506-265a-4465-a117-18174bb48c7d admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:25.638265 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1109/2219] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:12:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:12:25.638396 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027a864c-b263-44f2-ba47-8f0c3ec41522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:25.638923 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027a864c-b263-44f2-ba47-8f0c3ec41522 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:25.639188 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027a864c-b263-44f2-ba47-8f0c3ec41522 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:25.639457 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027a864c-b263-44f2-ba47-8f0c3ec41522 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:25.640124 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027a864c-b263-44f2-ba47-8f0c3ec41522 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:25.641810 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027a864c-b263-44f2-ba47-8f0c3ec41522 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:25.669852 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1111/2220] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:12:25 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 22:12:25.673251 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affa8b1f-bd71-4185-bb23-9ddea9343281 service neutron] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:25.691132 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affa8b1f-bd71-4185-bb23-9ddea9343281 service neutron] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:25.707115 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9b8ff6-bdb2-4787-88b1-7dca1a004721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:25.708331 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9b8ff6-bdb2-4787-88b1-7dca1a004721 admin admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:25.708644 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9b8ff6-bdb2-4787-88b1-7dca1a004721 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:25.708909 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9b8ff6-bdb2-4787-88b1-7dca1a004721 admin admin] PATH_INFO: `/v3/projects/e656860e14b044deb38ed079fcb5380a/users/fbbe8c7337ac4697b4a8dfbc0569943c/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:25.749412 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9b8ff6-bdb2-4787-88b1-7dca1a004721 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e656860e14b044deb38ed079fcb5380a, user_id=fbbe8c7337ac4697b4a8dfbc0569943c, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:25.752504 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9b8ff6-bdb2-4787-88b1-7dca1a004721 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:25.889956 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1110/2221] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:12:25 2022] PUT /identity/v3/projects/e656860e14b044deb38ed079fcb5380a/users/fbbe8c7337ac4697b4a8dfbc0569943c/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:12:25.894062 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027a864c-b263-44f2-ba47-8f0c3ec41522 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:25.911592 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027a864c-b263-44f2-ba47-8f0c3ec41522 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:26.007700 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed174c7-1275-4416-a195-62e293901f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:26.008716 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed174c7-1275-4416-a195-62e293901f2b admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:26.009070 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed174c7-1275-4416-a195-62e293901f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:26.009893 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed174c7-1275-4416-a195-62e293901f2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:26.011598 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed174c7-1275-4416-a195-62e293901f2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:26.016657 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed174c7-1275-4416-a195-62e293901f2b admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:26.031118 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1112/2222] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:12:25 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 22:12:26.035367 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9b8ff6-bdb2-4787-88b1-7dca1a004721 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:26.048195 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9b8ff6-bdb2-4787-88b1-7dca1a004721 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:26.053328 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac70c30e-b59c-46a3-aed7-d9eb3a4203b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:26.053973 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac70c30e-b59c-46a3-aed7-d9eb3a4203b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:26.054253 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac70c30e-b59c-46a3-aed7-d9eb3a4203b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:26.054530 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac70c30e-b59c-46a3-aed7-d9eb3a4203b8 admin admin] PATH_INFO: `/v3/projects/e656860e14b044deb38ed079fcb5380a/users/fbbe8c7337ac4697b4a8dfbc0569943c/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:26.056680 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac70c30e-b59c-46a3-aed7-d9eb3a4203b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e656860e14b044deb38ed079fcb5380a, user_id=fbbe8c7337ac4697b4a8dfbc0569943c, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:26.059614 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac70c30e-b59c-46a3-aed7-d9eb3a4203b8 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:26.070412 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1111/2223] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:12:26 2022] PUT /identity/v3/projects/e656860e14b044deb38ed079fcb5380a/users/fbbe8c7337ac4697b4a8dfbc0569943c/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:12:26.587007 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed174c7-1275-4416-a195-62e293901f2b admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:26.589466 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-bed174c7-1275-4416-a195-62e293901f2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:12:26.609521 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed174c7-1275-4416-a195-62e293901f2b admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:26.669975 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:12:26.769981 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5247a8-1b38-4589-a7ed-e707dc2aba23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:26.770774 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5247a8-1b38-4589-a7ed-e707dc2aba23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:26.771130 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5247a8-1b38-4589-a7ed-e707dc2aba23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:26.771508 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5247a8-1b38-4589-a7ed-e707dc2aba23 admin admin] PATH_INFO: `/v3/projects/4c307a8578514139b1d2e8d51fe75fed` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:26.774725 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5247a8-1b38-4589-a7ed-e707dc2aba23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c307a8578514139b1d2e8d51fe75fed)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:26.779756 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5247a8-1b38-4589-a7ed-e707dc2aba23 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:26.926336 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac70c30e-b59c-46a3-aed7-d9eb3a4203b8 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:26.928264 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-ac70c30e-b59c-46a3-aed7-d9eb3a4203b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:12:26.954328 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac70c30e-b59c-46a3-aed7-d9eb3a4203b8 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:26.961305 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6447bb98-0dcd-45a8-b0b7-53cb9550a9ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:26.962369 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6447bb98-0dcd-45a8-b0b7-53cb9550a9ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:26.962940 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6447bb98-0dcd-45a8-b0b7-53cb9550a9ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:26.963359 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6447bb98-0dcd-45a8-b0b7-53cb9550a9ed admin admin] PATH_INFO: `/v3/users/beef1fabdda74b3680ae544f3011e7e4` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:26.964991 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6447bb98-0dcd-45a8-b0b7-53cb9550a9ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=beef1fabdda74b3680ae544f3011e7e4)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:26.968977 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6447bb98-0dcd-45a8-b0b7-53cb9550a9ed admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:27.018835 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:12:27.028708 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-5e5247a8-1b38-4589-a7ed-e707dc2aba23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c307a8578514139b1d2e8d51fe75fed', 'request_id': 'req-5e5247a8-1b38-4589-a7ed-e707dc2aba23'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:27.235426 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-5e5247a8-1b38-4589-a7ed-e707dc2aba23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c307a8578514139b1d2e8d51fe75fed', 'request_id': 'req-5e5247a8-1b38-4589-a7ed-e707dc2aba23'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:27.247566 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-5e5247a8-1b38-4589-a7ed-e707dc2aba23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c307a8578514139b1d2e8d51fe75fed', 'request_id': 'req-5e5247a8-1b38-4589-a7ed-e707dc2aba23'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:27.264943 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-5e5247a8-1b38-4589-a7ed-e707dc2aba23 admin admin] The token cache is being invalidate because project 4c307a8578514139b1d2e8d51fe75fed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:12:27.265581 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-5e5247a8-1b38-4589-a7ed-e707dc2aba23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:27.266628 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-5e5247a8-1b38-4589-a7ed-e707dc2aba23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:27.268988 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1113/2224] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:12:26 2022] DELETE /identity/v3/projects/4c307a8578514139b1d2e8d51fe75fed => generated 0 bytes in 683 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:12:27.276238 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba688e35-ef6f-4e8c-8d3a-3f91320499c2 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:27.276772 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba688e35-ef6f-4e8c-8d3a-3f91320499c2 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:27.279245 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba688e35-ef6f-4e8c-8d3a-3f91320499c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:27.315526 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-ba688e35-ef6f-4e8c-8d3a-3f91320499c2 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:12:27.317060 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-ba688e35-ef6f-4e8c-8d3a-3f91320499c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:12:27.413879 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1114/2225] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:12:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:12:27.416665 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c063abeb-de86-4e39-972a-0cf1ebf7d076 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:27.418430 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c063abeb-de86-4e39-972a-0cf1ebf7d076 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:27.418923 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c063abeb-de86-4e39-972a-0cf1ebf7d076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:27.463847 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-6447bb98-0dcd-45a8-b0b7-53cb9550a9ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'beef1fabdda74b3680ae544f3011e7e4'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:27.465134 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-c063abeb-de86-4e39-972a-0cf1ebf7d076 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:12:27.468737 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-c063abeb-de86-4e39-972a-0cf1ebf7d076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:12:27.482464 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1115/2226] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:12:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:12:27.484590 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-6447bb98-0dcd-45a8-b0b7-53cb9550a9ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'beef1fabdda74b3680ae544f3011e7e4'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:27.487236 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c063abeb-de86-4e39-972a-0cf1ebf7d076 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:27.499726 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c063abeb-de86-4e39-972a-0cf1ebf7d076 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:27.506555 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca167d5-1faa-47e4-a5eb-ae3c60263bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:27.508002 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca167d5-1faa-47e4-a5eb-ae3c60263bee admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:27.508290 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca167d5-1faa-47e4-a5eb-ae3c60263bee admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:27.508618 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca167d5-1faa-47e4-a5eb-ae3c60263bee admin admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:27.509857 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca167d5-1faa-47e4-a5eb-ae3c60263bee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 22:12:27.510258 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca167d5-1faa-47e4-a5eb-ae3c60263bee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:27.512806 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca167d5-1faa-47e4-a5eb-ae3c60263bee admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:27.539067 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1116/2227] 10.222.0.18 () {58 vars in 1153 bytes} [Sat Aug 6 22:12:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 22:12:27.539994 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca167d5-1faa-47e4-a5eb-ae3c60263bee admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:27.565712 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca167d5-1faa-47e4-a5eb-ae3c60263bee admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:27.574369 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bd892d-1430-478e-a044-1bb3060e184e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:27.576013 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bd892d-1430-478e-a044-1bb3060e184e admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:27.576304 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bd892d-1430-478e-a044-1bb3060e184e admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:27.576632 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bd892d-1430-478e-a044-1bb3060e184e admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:27.577372 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bd892d-1430-478e-a044-1bb3060e184e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:27.579957 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bd892d-1430-478e-a044-1bb3060e184e admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:27.628739 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-6447bb98-0dcd-45a8-b0b7-53cb9550a9ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'beef1fabdda74b3680ae544f3011e7e4'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:27.648147 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-6447bb98-0dcd-45a8-b0b7-53cb9550a9ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'beef1fabdda74b3680ae544f3011e7e4'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:27.649004 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-6447bb98-0dcd-45a8-b0b7-53cb9550a9ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'beef1fabdda74b3680ae544f3011e7e4'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:27.660607 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1112/2228] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:12:26 2022] DELETE /identity/v3/users/beef1fabdda74b3680ae544f3011e7e4 => generated 0 bytes in 735 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 22:12:27.665199 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6246075-b46c-44ae-9888-52a385bb16f9 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:27.666122 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6246075-b46c-44ae-9888-52a385bb16f9 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:27.666514 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6246075-b46c-44ae-9888-52a385bb16f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:27.701881 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1117/2229] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:12:27 2022] POST /identity/v3/projects => generated 377 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:12:27.705299 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bd892d-1430-478e-a044-1bb3060e184e admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:27.710300 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-64bd892d-1430-478e-a044-1bb3060e184e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:12:27.735942 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-a6246075-b46c-44ae-9888-52a385bb16f9 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:12:27.737563 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-a6246075-b46c-44ae-9888-52a385bb16f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:12:27.863339 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bd892d-1430-478e-a044-1bb3060e184e admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:27.866006 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1113/2230] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:12:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 22:12:27.870223 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57eee85-ea82-483d-8589-fbacd78db3fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:27.870992 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57eee85-ea82-483d-8589-fbacd78db3fb admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:27.871478 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57eee85-ea82-483d-8589-fbacd78db3fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:27.871879 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57eee85-ea82-483d-8589-fbacd78db3fb admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:27.872756 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57eee85-ea82-483d-8589-fbacd78db3fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:27.875685 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6246075-b46c-44ae-9888-52a385bb16f9 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:27.876026 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57eee85-ea82-483d-8589-fbacd78db3fb admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:27.877394 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-a6246075-b46c-44ae-9888-52a385bb16f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:12:27.979277 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6246075-b46c-44ae-9888-52a385bb16f9 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:27.983176 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e00ffb-da09-4591-9bf1-465ab1c127e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:27.984008 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e00ffb-da09-4591-9bf1-465ab1c127e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:27.984347 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e00ffb-da09-4591-9bf1-465ab1c127e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:27.984717 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e00ffb-da09-4591-9bf1-465ab1c127e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:27.995091 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e00ffb-da09-4591-9bf1-465ab1c127e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:27.996935 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e00ffb-da09-4591-9bf1-465ab1c127e2 service neutron] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:28.008218 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1114/2231] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:12:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 22:12:28.088294 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1118/2232] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:12:27 2022] POST /identity/v3/users => generated 289 bytes in 384 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:12:28.090200 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e00ffb-da09-4591-9bf1-465ab1c127e2 service neutron] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:28.106667 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e00ffb-da09-4591-9bf1-465ab1c127e2 service neutron] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:28.113830 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee54878f-ee64-49c1-b03a-a4cf500fdfb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:28.114790 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee54878f-ee64-49c1-b03a-a4cf500fdfb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:28.115302 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee54878f-ee64-49c1-b03a-a4cf500fdfb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:28.115769 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee54878f-ee64-49c1-b03a-a4cf500fdfb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:28.116772 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee54878f-ee64-49c1-b03a-a4cf500fdfb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:28.118799 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee54878f-ee64-49c1-b03a-a4cf500fdfb7 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:28.136843 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57eee85-ea82-483d-8589-fbacd78db3fb admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:28.144008 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1115/2233] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:12:28 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:12:28.152419 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57eee85-ea82-483d-8589-fbacd78db3fb admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:28.170624 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee039588-9855-4434-9232-c19d66609868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:28.171265 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee039588-9855-4434-9232-c19d66609868 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:28.171547 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee039588-9855-4434-9232-c19d66609868 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:28.171822 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee039588-9855-4434-9232-c19d66609868 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:28.172521 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee039588-9855-4434-9232-c19d66609868 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:28.174183 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee039588-9855-4434-9232-c19d66609868 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:28.205268 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee54878f-ee64-49c1-b03a-a4cf500fdfb7 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:28.206292 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1119/2234] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:12:28 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:12:28.223669 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee54878f-ee64-49c1-b03a-a4cf500fdfb7 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:28.240640 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d877ac1c-326e-4f2d-912f-8976ff8d7d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:28.242394 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877ac1c-326e-4f2d-912f-8976ff8d7d3a admin admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:28.242888 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877ac1c-326e-4f2d-912f-8976ff8d7d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:28.243372 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877ac1c-326e-4f2d-912f-8976ff8d7d3a admin admin] PATH_INFO: `/v3/projects/d90478d3d3d64ea389f06f11dc2e5920/users/4e3318a3a0894427a937a9390867efe8/roles/56e4a96094e9434cbbafb22579c034f5` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:28.288195 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d877ac1c-326e-4f2d-912f-8976ff8d7d3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d90478d3d3d64ea389f06f11dc2e5920, user_id=4e3318a3a0894427a937a9390867efe8, role_id=56e4a96094e9434cbbafb22579c034f5)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:28.291596 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d877ac1c-326e-4f2d-912f-8976ff8d7d3a admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:28.382177 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:12:28.435026 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1116/2235] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:12:28 2022] PUT /identity/v3/projects/d90478d3d3d64ea389f06f11dc2e5920/users/4e3318a3a0894427a937a9390867efe8/roles/56e4a96094e9434cbbafb22579c034f5 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:12:28.438748 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221638e3-8997-425f-a9e9-3806a76b4213 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:28.439306 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221638e3-8997-425f-a9e9-3806a76b4213 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:28.439813 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221638e3-8997-425f-a9e9-3806a76b4213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:28.478389 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-221638e3-8997-425f-a9e9-3806a76b4213 None None] MFA Rules not processed for user `4e3318a3a0894427a937a9390867efe8`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:12:28.480344 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-221638e3-8997-425f-a9e9-3806a76b4213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:12:28.623255 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d877ac1c-326e-4f2d-912f-8976ff8d7d3a admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:28.630352 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1120/2236] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:12:28 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:12:28.718884 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d877ac1c-326e-4f2d-912f-8976ff8d7d3a admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:28.745529 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c85ec2-6ceb-4bb4-af92-6f35fa419b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:28.745529 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c85ec2-6ceb-4bb4-af92-6f35fa419b90 admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:28.753613 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c85ec2-6ceb-4bb4-af92-6f35fa419b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:28.753613 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c85ec2-6ceb-4bb4-af92-6f35fa419b90 admin admin] PATH_INFO: `/v3/projects` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:28.753613 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c85ec2-6ceb-4bb4-af92-6f35fa419b90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:28.753613 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c85ec2-6ceb-4bb4-af92-6f35fa419b90 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:28.931922 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1117/2237] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:12:28 2022] POST /identity/v3/projects => generated 377 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:12:28.936250 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221638e3-8997-425f-a9e9-3806a76b4213 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:28.966589 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221638e3-8997-425f-a9e9-3806a76b4213 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:29.121189 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901b542b-e346-49d6-b194-5c8fe2448385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:29.121899 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901b542b-e346-49d6-b194-5c8fe2448385 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:29.122139 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901b542b-e346-49d6-b194-5c8fe2448385 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:29.123548 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901b542b-e346-49d6-b194-5c8fe2448385 admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:29.124873 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901b542b-e346-49d6-b194-5c8fe2448385 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:29.129214 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901b542b-e346-49d6-b194-5c8fe2448385 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:29.344273 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c85ec2-6ceb-4bb4-af92-6f35fa419b90 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:29.351940 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-c8c85ec2-6ceb-4bb4-af92-6f35fa419b90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:12:29.360575 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1121/2238] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:12:28 2022] POST /identity/v3/users => generated 289 bytes in 425 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:12:29.364454 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901b542b-e346-49d6-b194-5c8fe2448385 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:29.390082 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c85ec2-6ceb-4bb4-af92-6f35fa419b90 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:29.394182 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901b542b-e346-49d6-b194-5c8fe2448385 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:29.397063 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7b8d46-2b7e-491f-9192-2f23cd6ac281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:29.397723 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7b8d46-2b7e-491f-9192-2f23cd6ac281 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:29.398006 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7b8d46-2b7e-491f-9192-2f23cd6ac281 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:29.398282 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7b8d46-2b7e-491f-9192-2f23cd6ac281 admin admin] PATH_INFO: `/v3/projects/6b6e295408f442b3b6c4b4da326531b4` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:29.405590 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd818f0-7da2-4dbc-9c6b-4a7c678d679b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:29.406378 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd818f0-7da2-4dbc-9c6b-4a7c678d679b admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:29.406682 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd818f0-7da2-4dbc-9c6b-4a7c678d679b admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:29.406928 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd818f0-7da2-4dbc-9c6b-4a7c678d679b admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:29.407347 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7b8d46-2b7e-491f-9192-2f23cd6ac281 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b6e295408f442b3b6c4b4da326531b4)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:29.407585 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd818f0-7da2-4dbc-9c6b-4a7c678d679b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:29.409369 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7b8d46-2b7e-491f-9192-2f23cd6ac281 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:29.413032 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd818f0-7da2-4dbc-9c6b-4a7c678d679b admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:29.436445 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1122/2239] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:12:29 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:12:29.439220 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd818f0-7da2-4dbc-9c6b-4a7c678d679b admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:29.456794 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd818f0-7da2-4dbc-9c6b-4a7c678d679b admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:29.473894 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a438c3a6-e576-452f-a2b6-794df708b5dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:29.474794 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a438c3a6-e576-452f-a2b6-794df708b5dd admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:29.475092 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a438c3a6-e576-452f-a2b6-794df708b5dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:29.475362 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a438c3a6-e576-452f-a2b6-794df708b5dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:29.476269 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a438c3a6-e576-452f-a2b6-794df708b5dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:29.479905 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a438c3a6-e576-452f-a2b6-794df708b5dd admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:29.504738 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1123/2240] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:12:29 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:12:29.508980 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a438c3a6-e576-452f-a2b6-794df708b5dd admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:29.524836 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a438c3a6-e576-452f-a2b6-794df708b5dd admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:29.533031 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee24502-f97f-4afa-b125-0cd63bb2e50f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:29.534631 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee24502-f97f-4afa-b125-0cd63bb2e50f admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:29.535398 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee24502-f97f-4afa-b125-0cd63bb2e50f admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:29.535865 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee24502-f97f-4afa-b125-0cd63bb2e50f admin admin] PATH_INFO: `/v3/projects/1cd8850e224c434cbb7368083e9829a3/users/bc268cb2bb1f4efe9848e4731488211f/roles/56e4a96094e9434cbbafb22579c034f5` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:29.581451 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee24502-f97f-4afa-b125-0cd63bb2e50f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cd8850e224c434cbb7368083e9829a3, user_id=bc268cb2bb1f4efe9848e4731488211f, role_id=56e4a96094e9434cbbafb22579c034f5)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:29.587000 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee24502-f97f-4afa-b125-0cd63bb2e50f admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:29.615616 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:12:29.618207 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-7a7b8d46-2b7e-491f-9192-2f23cd6ac281 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b6e295408f442b3b6c4b4da326531b4', 'request_id': 'req-7a7b8d46-2b7e-491f-9192-2f23cd6ac281'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:29.723799 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1124/2241] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:12:29 2022] PUT /identity/v3/projects/1cd8850e224c434cbb7368083e9829a3/users/bc268cb2bb1f4efe9848e4731488211f/roles/56e4a96094e9434cbbafb22579c034f5 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:12:29.725936 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303e8203-5dba-42f5-9465-3c6675bd2289 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:29.726415 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303e8203-5dba-42f5-9465-3c6675bd2289 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:29.727083 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303e8203-5dba-42f5-9465-3c6675bd2289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:29.768279 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-7a7b8d46-2b7e-491f-9192-2f23cd6ac281 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b6e295408f442b3b6c4b4da326531b4', 'request_id': 'req-7a7b8d46-2b7e-491f-9192-2f23cd6ac281'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:29.770742 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-7a7b8d46-2b7e-491f-9192-2f23cd6ac281 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b6e295408f442b3b6c4b4da326531b4', 'request_id': 'req-7a7b8d46-2b7e-491f-9192-2f23cd6ac281'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:29.817149 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-7a7b8d46-2b7e-491f-9192-2f23cd6ac281 admin admin] The token cache is being invalidate because project 6b6e295408f442b3b6c4b4da326531b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:12:29.817458 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-7a7b8d46-2b7e-491f-9192-2f23cd6ac281 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:29.818185 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-7a7b8d46-2b7e-491f-9192-2f23cd6ac281 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:29.821534 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1118/2242] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:12:29 2022] DELETE /identity/v3/projects/6b6e295408f442b3b6c4b4da326531b4 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:12:29.833253 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-303e8203-5dba-42f5-9465-3c6675bd2289 None None] MFA Rules not processed for user `bc268cb2bb1f4efe9848e4731488211f`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:12:29.834623 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-303e8203-5dba-42f5-9465-3c6675bd2289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:12:30.054867 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1125/2243] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:12:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:12:30.060532 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7b8d46-2b7e-491f-9192-2f23cd6ac281 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:30.062298 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-7a7b8d46-2b7e-491f-9192-2f23cd6ac281 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:12:30.224222 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7b8d46-2b7e-491f-9192-2f23cd6ac281 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:30.376378 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b682201-3c44-45e8-9ab5-5f491a11387a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:30.378839 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b682201-3c44-45e8-9ab5-5f491a11387a admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:30.399225 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b682201-3c44-45e8-9ab5-5f491a11387a admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:30.399225 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b682201-3c44-45e8-9ab5-5f491a11387a admin admin] PATH_INFO: `/v3/projects` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:30.404463 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b682201-3c44-45e8-9ab5-5f491a11387a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:30.410342 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b682201-3c44-45e8-9ab5-5f491a11387a admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:30.624531 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:12:30.627486 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686cb967-55fb-4978-a041-7090524a71a2 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:30.627779 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686cb967-55fb-4978-a041-7090524a71a2 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:30.628032 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686cb967-55fb-4978-a041-7090524a71a2 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:30.630042 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1126/2244] 10.222.0.18 () {54 vars in 921 bytes} [Sat Aug 6 22:12:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:12:30.640773 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1119/2245] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:12:30 2022] POST /identity/v3/projects => generated 375 bytes in 579 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:12:30.642839 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686cb967-55fb-4978-a041-7090524a71a2 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:30.687801 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36641da7-0359-4546-baa8-de47ad9ba65a None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:30.688364 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36641da7-0359-4546-baa8-de47ad9ba65a None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:30.688364 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36641da7-0359-4546-baa8-de47ad9ba65a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:30.722495 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-36641da7-0359-4546-baa8-de47ad9ba65a None None] MFA Rules not processed for user `680d62a67ae2456f8c129e26728c3297`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:12:30.722495 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-36641da7-0359-4546-baa8-de47ad9ba65a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:12:30.764306 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686cb967-55fb-4978-a041-7090524a71a2 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:30.769685 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55f5f21-5ca6-4904-8729-b63e0801c7d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:30.770310 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55f5f21-5ca6-4904-8729-b63e0801c7d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:30.770558 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55f5f21-5ca6-4904-8729-b63e0801c7d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:30.770828 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55f5f21-5ca6-4904-8729-b63e0801c7d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:30.771455 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55f5f21-5ca6-4904-8729-b63e0801c7d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:30.774252 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55f5f21-5ca6-4904-8729-b63e0801c7d7 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:30.823576 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1120/2246] 10.222.0.18 () {58 vars in 1018 bytes} [Sat Aug 6 22:12:30 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:12:30.849069 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36641da7-0359-4546-baa8-de47ad9ba65a None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:30.850138 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-36641da7-0359-4546-baa8-de47ad9ba65a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:12:30.878876 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36641da7-0359-4546-baa8-de47ad9ba65a None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:30.894920 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1127/2247] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:12:30 2022] POST /identity/v3/users => generated 294 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:12:30.894920 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55f5f21-5ca6-4904-8729-b63e0801c7d7 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:30.894920 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-b55f5f21-5ca6-4904-8729-b63e0801c7d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:12:30.895441 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d36a77-e35f-4205-8d68-8bc05a211963 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '680d62a67ae2456f8c129e26728c3297', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:30.896150 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d36a77-e35f-4205-8d68-8bc05a211963 service placement] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:30.896594 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d36a77-e35f-4205-8d68-8bc05a211963 service placement] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:30.897210 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d36a77-e35f-4205-8d68-8bc05a211963 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:30.916521 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d36a77-e35f-4205-8d68-8bc05a211963 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:30.920626 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d36a77-e35f-4205-8d68-8bc05a211963 service placement] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:30.922380 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55f5f21-5ca6-4904-8729-b63e0801c7d7 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:30.938592 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c786cf8-bad1-4805-9d78-fee3b103ee95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:30.939500 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c786cf8-bad1-4805-9d78-fee3b103ee95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:30.942117 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c786cf8-bad1-4805-9d78-fee3b103ee95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:30.942557 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c786cf8-bad1-4805-9d78-fee3b103ee95 admin admin] PATH_INFO: `/v3/users/13f161eb377e4f809b2232c1047c0bb5` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:30.947152 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c786cf8-bad1-4805-9d78-fee3b103ee95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13f161eb377e4f809b2232c1047c0bb5)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:30.951197 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c786cf8-bad1-4805-9d78-fee3b103ee95 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:30.953348 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1121/2248] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:12:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:12:30.955240 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d36a77-e35f-4205-8d68-8bc05a211963 service placement] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:30.980099 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d36a77-e35f-4205-8d68-8bc05a211963 service placement] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:30.989619 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a968c2a1-7434-4985-8938-e0ea01b2df9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:30.991606 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a968c2a1-7434-4985-8938-e0ea01b2df9e admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:30.996519 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a968c2a1-7434-4985-8938-e0ea01b2df9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:30.996519 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a968c2a1-7434-4985-8938-e0ea01b2df9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:30.996519 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a968c2a1-7434-4985-8938-e0ea01b2df9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:31.001522 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a968c2a1-7434-4985-8938-e0ea01b2df9e admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:31.021069 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1122/2249] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:12:30 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:12:31.022539 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a968c2a1-7434-4985-8938-e0ea01b2df9e admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:31.035594 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a968c2a1-7434-4985-8938-e0ea01b2df9e admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:31.044563 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67907709-16d8-44dc-9636-423c9e9110c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:31.045323 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67907709-16d8-44dc-9636-423c9e9110c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:31.045669 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67907709-16d8-44dc-9636-423c9e9110c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:31.046162 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67907709-16d8-44dc-9636-423c9e9110c5 admin admin] PATH_INFO: `/v3/projects/7d3f60a3f7474a1ab0e3cc1f2d18e7e3/users/ace9b1ecc017421285a0653dd9f4d2db/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:31.066962 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67907709-16d8-44dc-9636-423c9e9110c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d3f60a3f7474a1ab0e3cc1f2d18e7e3, user_id=ace9b1ecc017421285a0653dd9f4d2db, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:31.071219 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67907709-16d8-44dc-9636-423c9e9110c5 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:31.173323 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1123/2250] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:12:31 2022] PUT /identity/v3/projects/7d3f60a3f7474a1ab0e3cc1f2d18e7e3/users/ace9b1ecc017421285a0653dd9f4d2db/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 22:12:31.173323 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67907709-16d8-44dc-9636-423c9e9110c5 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:31.224954 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-1c786cf8-bad1-4805-9d78-fee3b103ee95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13f161eb377e4f809b2232c1047c0bb5'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:31.272521 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-1c786cf8-bad1-4805-9d78-fee3b103ee95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13f161eb377e4f809b2232c1047c0bb5'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:31.274359 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67907709-16d8-44dc-9636-423c9e9110c5 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:31.283668 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-1c786cf8-bad1-4805-9d78-fee3b103ee95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13f161eb377e4f809b2232c1047c0bb5'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:31.284389 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-1c786cf8-bad1-4805-9d78-fee3b103ee95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13f161eb377e4f809b2232c1047c0bb5'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:31.285252 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2313b50b-9522-4ff4-9337-4e58c1902d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:31.285408 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2313b50b-9522-4ff4-9337-4e58c1902d1f admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:31.285669 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-1c786cf8-bad1-4805-9d78-fee3b103ee95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13f161eb377e4f809b2232c1047c0bb5'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:31.285779 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2313b50b-9522-4ff4-9337-4e58c1902d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:31.285928 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2313b50b-9522-4ff4-9337-4e58c1902d1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:31.286584 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2313b50b-9522-4ff4-9337-4e58c1902d1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:31.289234 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2313b50b-9522-4ff4-9337-4e58c1902d1f admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:31.309284 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1124/2251] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:12:31 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:12:31.313077 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2313b50b-9522-4ff4-9337-4e58c1902d1f admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:31.316472 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-2313b50b-9522-4ff4-9337-4e58c1902d1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:12:31.335105 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2313b50b-9522-4ff4-9337-4e58c1902d1f admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:31.344850 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8705fdd-0255-4dd9-91a5-b5d1e4475a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:31.344850 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8705fdd-0255-4dd9-91a5-b5d1e4475a7b admin admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:31.344850 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8705fdd-0255-4dd9-91a5-b5d1e4475a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:31.348182 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8705fdd-0255-4dd9-91a5-b5d1e4475a7b admin admin] PATH_INFO: `/v3/projects/7d3f60a3f7474a1ab0e3cc1f2d18e7e3/users/ace9b1ecc017421285a0653dd9f4d2db/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:31.353281 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8705fdd-0255-4dd9-91a5-b5d1e4475a7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d3f60a3f7474a1ab0e3cc1f2d18e7e3, user_id=ace9b1ecc017421285a0653dd9f4d2db, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:31.359763 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8705fdd-0255-4dd9-91a5-b5d1e4475a7b admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:31.386199 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1128/2252] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:12:30 2022] DELETE /identity/v3/users/13f161eb377e4f809b2232c1047c0bb5 => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:12:31.404891 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1125/2253] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:12:31 2022] PUT /identity/v3/projects/7d3f60a3f7474a1ab0e3cc1f2d18e7e3/users/ace9b1ecc017421285a0653dd9f4d2db/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:12:31.418770 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e5feb-6e1e-42f9-b7dc-8ce428c047c7 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:31.418989 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e5feb-6e1e-42f9-b7dc-8ce428c047c7 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:31.419250 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e5feb-6e1e-42f9-b7dc-8ce428c047c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:31.454876 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-581e5feb-6e1e-42f9-b7dc-8ce428c047c7 None None] MFA Rules not processed for user `ace9b1ecc017421285a0653dd9f4d2db`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:12:31.456129 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-581e5feb-6e1e-42f9-b7dc-8ce428c047c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:12:31.550194 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1129/2254] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:12:31 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:12:31.566343 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8705fdd-0255-4dd9-91a5-b5d1e4475a7b admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:31.569124 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-f8705fdd-0255-4dd9-91a5-b5d1e4475a7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:12:31.698332 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8705fdd-0255-4dd9-91a5-b5d1e4475a7b admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:31.703244 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bc1b45-42f6-41b8-a396-c7b59766064a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa71c71de263478da234f3c5716a6e5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:31.704046 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc1b45-42f6-41b8-a396-c7b59766064a service cinder] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:31.704628 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc1b45-42f6-41b8-a396-c7b59766064a service cinder] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:31.705003 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc1b45-42f6-41b8-a396-c7b59766064a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:31.711456 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-16bc1b45-42f6-41b8-a396-c7b59766064a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:12:31.792267 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bc1b45-42f6-41b8-a396-c7b59766064a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:31.794120 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bc1b45-42f6-41b8-a396-c7b59766064a service cinder] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:31.811468 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1126/2255] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:12:31 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:12:31.882709 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581e5feb-6e1e-42f9-b7dc-8ce428c047c7 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:31.885532 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-581e5feb-6e1e-42f9-b7dc-8ce428c047c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:12:32.010685 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:12:32.055164 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581e5feb-6e1e-42f9-b7dc-8ce428c047c7 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:32.060068 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d54354-dc51-4839-bb6c-512a3ecfd2ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:32.069159 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d54354-dc51-4839-bb6c-512a3ecfd2ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:32.069159 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d54354-dc51-4839-bb6c-512a3ecfd2ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:32.069159 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d54354-dc51-4839-bb6c-512a3ecfd2ea admin admin] PATH_INFO: `/v3/projects/b1386a21549b4faf87836a9be75c670c` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:32.069159 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d54354-dc51-4839-bb6c-512a3ecfd2ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1386a21549b4faf87836a9be75c670c)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:32.069857 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d54354-dc51-4839-bb6c-512a3ecfd2ea admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:32.212990 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-a8d54354-dc51-4839-bb6c-512a3ecfd2ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1386a21549b4faf87836a9be75c670c', 'request_id': 'req-a8d54354-dc51-4839-bb6c-512a3ecfd2ea'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:32.326946 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=90,keystone:INSERT=4,keystone:DELETE=6 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:12:32.330645 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-a8d54354-dc51-4839-bb6c-512a3ecfd2ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1386a21549b4faf87836a9be75c670c', 'request_id': 'req-a8d54354-dc51-4839-bb6c-512a3ecfd2ea'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:32.338675 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-a8d54354-dc51-4839-bb6c-512a3ecfd2ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1386a21549b4faf87836a9be75c670c', 'request_id': 'req-a8d54354-dc51-4839-bb6c-512a3ecfd2ea'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:32.354842 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-a8d54354-dc51-4839-bb6c-512a3ecfd2ea admin admin] The token cache is being invalidate because project b1386a21549b4faf87836a9be75c670c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:12:32.355166 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-a8d54354-dc51-4839-bb6c-512a3ecfd2ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:32.355918 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-a8d54354-dc51-4839-bb6c-512a3ecfd2ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:32.357734 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1130/2256] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:12:31 2022] DELETE /identity/v3/projects/b1386a21549b4faf87836a9be75c670c => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:12:32.367406 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53501c29-8c93-4851-9ab5-b2ec6f7968a6 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:32.367406 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53501c29-8c93-4851-9ab5-b2ec6f7968a6 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:32.367406 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53501c29-8c93-4851-9ab5-b2ec6f7968a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:32.383808 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-53501c29-8c93-4851-9ab5-b2ec6f7968a6 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:12:32.385856 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-53501c29-8c93-4851-9ab5-b2ec6f7968a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:12:32.495685 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1127/2257] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:12:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:12:32.503820 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69476206-a815-4f39-a470-0e21b0d7b00a None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:32.504109 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69476206-a815-4f39-a470-0e21b0d7b00a None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:32.504388 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69476206-a815-4f39-a470-0e21b0d7b00a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:32.535924 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-69476206-a815-4f39-a470-0e21b0d7b00a None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:12:32.537214 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-69476206-a815-4f39-a470-0e21b0d7b00a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:12:32.554820 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1131/2258] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:12:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:12:32.561461 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53501c29-8c93-4851-9ab5-b2ec6f7968a6 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:32.566786 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53501c29-8c93-4851-9ab5-b2ec6f7968a6 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:32.573933 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c40afb8-4574-4b6b-9512-2f787bd03281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:32.574800 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c40afb8-4574-4b6b-9512-2f787bd03281 admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:32.575232 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c40afb8-4574-4b6b-9512-2f787bd03281 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:32.575652 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c40afb8-4574-4b6b-9512-2f787bd03281 admin admin] PATH_INFO: `/v3/domains` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:32.576529 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c40afb8-4574-4b6b-9512-2f787bd03281 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 22:12:32.577007 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c40afb8-4574-4b6b-9512-2f787bd03281 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:32.579330 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c40afb8-4574-4b6b-9512-2f787bd03281 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:32.607341 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69476206-a815-4f39-a470-0e21b0d7b00a None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:32.607452 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1128/2259] 10.222.0.18 () {58 vars in 1153 bytes} [Sat Aug 6 22:12:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 22:12:32.611766 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69476206-a815-4f39-a470-0e21b0d7b00a None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:32.623348 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abad030-a8ef-480c-9f73-4e11a3953983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:32.624062 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abad030-a8ef-480c-9f73-4e11a3953983 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:32.624422 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abad030-a8ef-480c-9f73-4e11a3953983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:32.625027 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abad030-a8ef-480c-9f73-4e11a3953983 admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:32.626410 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abad030-a8ef-480c-9f73-4e11a3953983 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:32.629016 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abad030-a8ef-480c-9f73-4e11a3953983 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:32.825372 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1132/2260] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:12:32 2022] POST /identity/v3/projects => generated 393 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 22:12:32.828393 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c40afb8-4574-4b6b-9512-2f787bd03281 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:32.841577 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c40afb8-4574-4b6b-9512-2f787bd03281 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:32.926874 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1215b0-f2cf-4d23-aa24-9f235c534414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:32.928393 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1215b0-f2cf-4d23-aa24-9f235c534414 admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:32.934351 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1215b0-f2cf-4d23-aa24-9f235c534414 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:32.934351 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1215b0-f2cf-4d23-aa24-9f235c534414 admin admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:32.934351 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1215b0-f2cf-4d23-aa24-9f235c534414 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:32.934351 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1215b0-f2cf-4d23-aa24-9f235c534414 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:33.043546 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1129/2261] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:12:32 2022] POST /identity/v3/users => generated 297 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:12:33.046507 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abad030-a8ef-480c-9f73-4e11a3953983 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:33.067802 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abad030-a8ef-480c-9f73-4e11a3953983 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:33.116803 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c138e8b4-9c01-4dd8-baa1-5c77dd433f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:33.119510 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138e8b4-9c01-4dd8-baa1-5c77dd433f50 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:33.119877 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138e8b4-9c01-4dd8-baa1-5c77dd433f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:33.120229 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138e8b4-9c01-4dd8-baa1-5c77dd433f50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:33.120932 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c138e8b4-9c01-4dd8-baa1-5c77dd433f50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:33.130739 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c138e8b4-9c01-4dd8-baa1-5c77dd433f50 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:33.159264 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1133/2262] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:12:33 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:12:33.163541 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1215b0-f2cf-4d23-aa24-9f235c534414 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:33.181559 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1215b0-f2cf-4d23-aa24-9f235c534414 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:33.187952 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f143bf97-a813-4e8b-bb11-1077f14e226c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:33.188583 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f143bf97-a813-4e8b-bb11-1077f14e226c admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:33.188864 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f143bf97-a813-4e8b-bb11-1077f14e226c admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:33.189136 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f143bf97-a813-4e8b-bb11-1077f14e226c admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:33.189828 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f143bf97-a813-4e8b-bb11-1077f14e226c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:33.191553 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f143bf97-a813-4e8b-bb11-1077f14e226c admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:33.215124 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1130/2263] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:12:33 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 22:12:33.219932 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c138e8b4-9c01-4dd8-baa1-5c77dd433f50 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:33.234681 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c138e8b4-9c01-4dd8-baa1-5c77dd433f50 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:33.246748 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdc9d02-dd2b-41b2-857a-d0ef5045401e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:33.248129 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdc9d02-dd2b-41b2-857a-d0ef5045401e admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:33.248129 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdc9d02-dd2b-41b2-857a-d0ef5045401e admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:33.248316 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdc9d02-dd2b-41b2-857a-d0ef5045401e admin admin] PATH_INFO: `/v3/projects/480896a54dab43c8980c8b05c9a793e7/users/700f40b4567a4bcdbb2194861f4497d5/roles/56e4a96094e9434cbbafb22579c034f5` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:33.281150 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdc9d02-dd2b-41b2-857a-d0ef5045401e admin admin] RBAC: Authorizing `identity:create_grant(project_id=480896a54dab43c8980c8b05c9a793e7, user_id=700f40b4567a4bcdbb2194861f4497d5, role_id=56e4a96094e9434cbbafb22579c034f5)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:33.284706 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdc9d02-dd2b-41b2-857a-d0ef5045401e admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:33.411969 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1134/2264] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:12:33 2022] PUT /identity/v3/projects/480896a54dab43c8980c8b05c9a793e7/users/700f40b4567a4bcdbb2194861f4497d5/roles/56e4a96094e9434cbbafb22579c034f5 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 22:12:33.418186 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82da36b-4c1a-40ac-9a37-7555b0388071 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:33.418472 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82da36b-4c1a-40ac-9a37-7555b0388071 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:33.419211 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82da36b-4c1a-40ac-9a37-7555b0388071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:33.440549 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-b82da36b-4c1a-40ac-9a37-7555b0388071 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:12:33.441776 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-b82da36b-4c1a-40ac-9a37-7555b0388071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:12:33.517338 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1131/2265] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:12:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:12:33.537564 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdc9d02-dd2b-41b2-857a-d0ef5045401e admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:33.539480 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-2cdc9d02-dd2b-41b2-857a-d0ef5045401e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:12:33.669585 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdc9d02-dd2b-41b2-857a-d0ef5045401e admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:33.674286 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa8dbc3-2b22-48b6-8d78-814f08b2a89b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:33.675756 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa8dbc3-2b22-48b6-8d78-814f08b2a89b service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:33.675756 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa8dbc3-2b22-48b6-8d78-814f08b2a89b service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:33.675756 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa8dbc3-2b22-48b6-8d78-814f08b2a89b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:33.685889 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa8dbc3-2b22-48b6-8d78-814f08b2a89b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:33.687387 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa8dbc3-2b22-48b6-8d78-814f08b2a89b service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:33.698015 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1135/2266] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:12:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:12:34.428283 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:12:34.748174 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:12:35.261478 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35501cf-830f-4f44-9b55-d8039968d6ce None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:35.262404 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35501cf-830f-4f44-9b55-d8039968d6ce None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:35.262630 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35501cf-830f-4f44-9b55-d8039968d6ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:35.312111 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-b35501cf-830f-4f44-9b55-d8039968d6ce None None] MFA Rules not processed for user `fbbe8c7337ac4697b4a8dfbc0569943c`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:12:35.313437 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-b35501cf-830f-4f44-9b55-d8039968d6ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:12:35.422954 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa8dbc3-2b22-48b6-8d78-814f08b2a89b service neutron] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:35.422954 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-caa8dbc3-2b22-48b6-8d78-814f08b2a89b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:12:35.424869 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1132/2267] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:12:35 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:12:35.439434 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35501cf-830f-4f44-9b55-d8039968d6ce None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:35.441037 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-b35501cf-830f-4f44-9b55-d8039968d6ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:12:35.508335 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa8dbc3-2b22-48b6-8d78-814f08b2a89b service neutron] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:35.513528 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f967d04-4178-4db2-80ac-fb079c0ed749 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa71c71de263478da234f3c5716a6e5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:35.516008 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f967d04-4178-4db2-80ac-fb079c0ed749 service cinder] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:35.516008 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f967d04-4178-4db2-80ac-fb079c0ed749 service cinder] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:35.516008 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f967d04-4178-4db2-80ac-fb079c0ed749 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:35.517197 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-7f967d04-4178-4db2-80ac-fb079c0ed749 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:12:35.523359 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35501cf-830f-4f44-9b55-d8039968d6ce None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:35.541576 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e66e4c-f441-4b17-8440-b9fc8cf0eca8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a1319d33e9144859c6010843be312a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:35.542394 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e66e4c-f441-4b17-8440-b9fc8cf0eca8 service nova] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:35.542838 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e66e4c-f441-4b17-8440-b9fc8cf0eca8 service nova] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:35.543289 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e66e4c-f441-4b17-8440-b9fc8cf0eca8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:35.547484 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-e0e66e4c-f441-4b17-8440-b9fc8cf0eca8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:12:35.610251 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f967d04-4178-4db2-80ac-fb079c0ed749 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:35.611725 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f967d04-4178-4db2-80ac-fb079c0ed749 service cinder] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:35.622436 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1136/2268] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:12:35 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:12:35.674484 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e66e4c-f441-4b17-8440-b9fc8cf0eca8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:35.676695 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e66e4c-f441-4b17-8440-b9fc8cf0eca8 service nova] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:35.700676 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1133/2269] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:12:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:12:35.806574 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:12:37.720489 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:12:37.954723 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f967d04-4178-4db2-80ac-fb079c0ed749 service cinder] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:37.964175 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f967d04-4178-4db2-80ac-fb079c0ed749 service cinder] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:37.967925 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a43fa1-6dbf-44fa-8b96-1075cffcd767 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa71c71de263478da234f3c5716a6e5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:37.968873 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a43fa1-6dbf-44fa-8b96-1075cffcd767 service cinder] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:37.969301 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a43fa1-6dbf-44fa-8b96-1075cffcd767 service cinder] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:37.969706 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a43fa1-6dbf-44fa-8b96-1075cffcd767 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:37.971785 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-08a43fa1-6dbf-44fa-8b96-1075cffcd767 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:12:38.058761 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a43fa1-6dbf-44fa-8b96-1075cffcd767 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:38.060801 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a43fa1-6dbf-44fa-8b96-1075cffcd767 service cinder] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:38.107770 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1137/2270] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:12:37 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:12:38.422644 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:12:40.307622 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7c5c25-01da-4153-a06e-b621e162ae1d None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:40.308694 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7c5c25-01da-4153-a06e-b621e162ae1d None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:40.309170 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7c5c25-01da-4153-a06e-b621e162ae1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:40.334071 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-fb7c5c25-01da-4153-a06e-b621e162ae1d None None] MFA Rules not processed for user `9469cd012c2549288ba403902b667295`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:12:40.340926 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-fb7c5c25-01da-4153-a06e-b621e162ae1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:12:40.369588 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1134/2271] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:12:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:12:41.279670 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfcac8e-d79a-4035-95ff-93dec94b1312 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:41.280130 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfcac8e-d79a-4035-95ff-93dec94b1312 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:41.280130 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfcac8e-d79a-4035-95ff-93dec94b1312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:41.311518 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-1bfcac8e-d79a-4035-95ff-93dec94b1312 None None] MFA Rules not processed for user `700f40b4567a4bcdbb2194861f4497d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:12:41.312786 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-1bfcac8e-d79a-4035-95ff-93dec94b1312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:12:41.421707 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be360d85-51c0-4faa-91d2-eb62d1690f85 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:41.423369 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be360d85-51c0-4faa-91d2-eb62d1690f85 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:41.424113 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be360d85-51c0-4faa-91d2-eb62d1690f85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:41.427102 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1138/2272] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:12:41 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:12:41.453069 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-be360d85-51c0-4faa-91d2-eb62d1690f85 None None] MFA Rules not processed for user `700f40b4567a4bcdbb2194861f4497d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:12:41.454303 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-be360d85-51c0-4faa-91d2-eb62d1690f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:12:41.473797 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1135/2273] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:12:41 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:12:41.480708 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfcac8e-d79a-4035-95ff-93dec94b1312 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:41.518938 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfcac8e-d79a-4035-95ff-93dec94b1312 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:41.526959 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10022aa3-090d-4a8e-a584-b13fef41b419 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a1319d33e9144859c6010843be312a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:41.527589 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10022aa3-090d-4a8e-a584-b13fef41b419 service nova] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:41.527861 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10022aa3-090d-4a8e-a584-b13fef41b419 service nova] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:41.528235 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10022aa3-090d-4a8e-a584-b13fef41b419 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:41.539211 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10022aa3-090d-4a8e-a584-b13fef41b419 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:41.542252 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10022aa3-090d-4a8e-a584-b13fef41b419 service nova] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:41.562782 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1139/2274] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:12:41 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:12:42.691948 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39758cf-d9ad-4489-a2ed-552bcf6667fe None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:42.692462 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39758cf-d9ad-4489-a2ed-552bcf6667fe None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:42.692711 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39758cf-d9ad-4489-a2ed-552bcf6667fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:42.717550 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-a39758cf-d9ad-4489-a2ed-552bcf6667fe None None] MFA Rules not processed for user `700f40b4567a4bcdbb2194861f4497d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:12:42.718755 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-a39758cf-d9ad-4489-a2ed-552bcf6667fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:12:42.728279 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1136/2275] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:12:42 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:12:44.558946 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10022aa3-090d-4a8e-a584-b13fef41b419 service nova] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:44.561229 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-10022aa3-090d-4a8e-a584-b13fef41b419 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:12:44.582658 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10022aa3-090d-4a8e-a584-b13fef41b419 service nova] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:44.592878 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f271d19-d894-47c9-91e4-17303cbba44b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:44.593684 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f271d19-d894-47c9-91e4-17303cbba44b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:44.594000 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f271d19-d894-47c9-91e4-17303cbba44b admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:44.594283 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f271d19-d894-47c9-91e4-17303cbba44b admin admin] PATH_INFO: `/v3/users/4e3318a3a0894427a937a9390867efe8` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:44.596050 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f271d19-d894-47c9-91e4-17303cbba44b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e3318a3a0894427a937a9390867efe8)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:44.598424 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f271d19-d894-47c9-91e4-17303cbba44b admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:44.904921 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:12:44.918179 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-4f271d19-d894-47c9-91e4-17303cbba44b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e3318a3a0894427a937a9390867efe8'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:44.931615 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-4f271d19-d894-47c9-91e4-17303cbba44b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e3318a3a0894427a937a9390867efe8'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:44.942336 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-4f271d19-d894-47c9-91e4-17303cbba44b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e3318a3a0894427a937a9390867efe8'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:44.943634 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-4f271d19-d894-47c9-91e4-17303cbba44b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e3318a3a0894427a937a9390867efe8'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:44.944474 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-4f271d19-d894-47c9-91e4-17303cbba44b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e3318a3a0894427a937a9390867efe8'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:45.068055 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1140/2276] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:12:44 2022] DELETE /identity/v3/users/4e3318a3a0894427a937a9390867efe8 => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 22:12:45.068150 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbd0d3a-7979-489b-85b6-d8f0418d111c None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:45.068150 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbd0d3a-7979-489b-85b6-d8f0418d111c None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:45.068648 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbd0d3a-7979-489b-85b6-d8f0418d111c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:45.088217 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-efbd0d3a-7979-489b-85b6-d8f0418d111c None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:12:45.089448 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-efbd0d3a-7979-489b-85b6-d8f0418d111c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:12:45.168070 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1137/2277] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:12:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:12:45.177510 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f271d19-d894-47c9-91e4-17303cbba44b admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:45.178960 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-4f271d19-d894-47c9-91e4-17303cbba44b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:12:45.250532 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f271d19-d894-47c9-91e4-17303cbba44b admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:45.254867 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21019303-4d0f-400d-9876-162fa3220f7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:45.255898 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21019303-4d0f-400d-9876-162fa3220f7f service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:45.256252 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21019303-4d0f-400d-9876-162fa3220f7f service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:45.256620 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21019303-4d0f-400d-9876-162fa3220f7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:45.267162 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21019303-4d0f-400d-9876-162fa3220f7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:45.268730 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21019303-4d0f-400d-9876-162fa3220f7f service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:45.284304 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1141/2278] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:12:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:12:45.823829 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbd0d3a-7979-489b-85b6-d8f0418d111c None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:45.824037 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-efbd0d3a-7979-489b-85b6-d8f0418d111c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:12:45.844148 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbd0d3a-7979-489b-85b6-d8f0418d111c None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:45.850013 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690fe169-206a-4695-89a2-277a85b92a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:45.851293 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690fe169-206a-4695-89a2-277a85b92a22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:45.851893 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690fe169-206a-4695-89a2-277a85b92a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:45.853231 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690fe169-206a-4695-89a2-277a85b92a22 admin admin] PATH_INFO: `/v3/projects/d90478d3d3d64ea389f06f11dc2e5920` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:45.855236 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690fe169-206a-4695-89a2-277a85b92a22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d90478d3d3d64ea389f06f11dc2e5920)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:45.857342 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690fe169-206a-4695-89a2-277a85b92a22 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:46.060390 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:12:46.069488 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-690fe169-206a-4695-89a2-277a85b92a22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd90478d3d3d64ea389f06f11dc2e5920', 'request_id': 'req-690fe169-206a-4695-89a2-277a85b92a22'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:46.214870 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-690fe169-206a-4695-89a2-277a85b92a22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd90478d3d3d64ea389f06f11dc2e5920', 'request_id': 'req-690fe169-206a-4695-89a2-277a85b92a22'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:46.220999 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-690fe169-206a-4695-89a2-277a85b92a22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd90478d3d3d64ea389f06f11dc2e5920', 'request_id': 'req-690fe169-206a-4695-89a2-277a85b92a22'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:46.233264 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-690fe169-206a-4695-89a2-277a85b92a22 admin admin] The token cache is being invalidate because project d90478d3d3d64ea389f06f11dc2e5920 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:12:46.233536 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-690fe169-206a-4695-89a2-277a85b92a22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:46.234177 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-690fe169-206a-4695-89a2-277a85b92a22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:46.235660 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1138/2279] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:12:45 2022] DELETE /identity/v3/projects/d90478d3d3d64ea389f06f11dc2e5920 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:12:46.239821 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21019303-4d0f-400d-9876-162fa3220f7f service neutron] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:46.250426 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21019303-4d0f-400d-9876-162fa3220f7f service neutron] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:46.312577 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9d5c39-3276-44cf-a9d5-23a1ed2055c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:46.313131 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d5c39-3276-44cf-a9d5-23a1ed2055c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:46.313412 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d5c39-3276-44cf-a9d5-23a1ed2055c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:46.313687 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d5c39-3276-44cf-a9d5-23a1ed2055c7 admin admin] PATH_INFO: `/v3/users/bc268cb2bb1f4efe9848e4731488211f` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:46.314829 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9d5c39-3276-44cf-a9d5-23a1ed2055c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc268cb2bb1f4efe9848e4731488211f)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:46.316868 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9d5c39-3276-44cf-a9d5-23a1ed2055c7 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:46.542148 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-fd9d5c39-3276-44cf-a9d5-23a1ed2055c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc268cb2bb1f4efe9848e4731488211f'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:46.554012 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-fd9d5c39-3276-44cf-a9d5-23a1ed2055c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc268cb2bb1f4efe9848e4731488211f'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:46.560949 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-fd9d5c39-3276-44cf-a9d5-23a1ed2055c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc268cb2bb1f4efe9848e4731488211f'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:46.561641 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-fd9d5c39-3276-44cf-a9d5-23a1ed2055c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc268cb2bb1f4efe9848e4731488211f'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:46.562260 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-fd9d5c39-3276-44cf-a9d5-23a1ed2055c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc268cb2bb1f4efe9848e4731488211f'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:46.667115 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1142/2280] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:12:46 2022] DELETE /identity/v3/users/bc268cb2bb1f4efe9848e4731488211f => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:12:47.386297 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690fe169-206a-4695-89a2-277a85b92a22 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:47.389068 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-690fe169-206a-4695-89a2-277a85b92a22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:12:47.411656 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690fe169-206a-4695-89a2-277a85b92a22 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:47.481103 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ff5bc5-282b-47d9-8a7a-09e4d78948d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:47.481775 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ff5bc5-282b-47d9-8a7a-09e4d78948d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:47.482054 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ff5bc5-282b-47d9-8a7a-09e4d78948d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:47.482331 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ff5bc5-282b-47d9-8a7a-09e4d78948d3 admin admin] PATH_INFO: `/v3/projects/1cd8850e224c434cbb7368083e9829a3` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:47.483577 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ff5bc5-282b-47d9-8a7a-09e4d78948d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cd8850e224c434cbb7368083e9829a3)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:47.485560 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ff5bc5-282b-47d9-8a7a-09e4d78948d3 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:47.629275 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-89ff5bc5-282b-47d9-8a7a-09e4d78948d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cd8850e224c434cbb7368083e9829a3', 'request_id': 'req-89ff5bc5-282b-47d9-8a7a-09e4d78948d3'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:47.746293 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-89ff5bc5-282b-47d9-8a7a-09e4d78948d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cd8850e224c434cbb7368083e9829a3', 'request_id': 'req-89ff5bc5-282b-47d9-8a7a-09e4d78948d3'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:47.754985 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-89ff5bc5-282b-47d9-8a7a-09e4d78948d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cd8850e224c434cbb7368083e9829a3', 'request_id': 'req-89ff5bc5-282b-47d9-8a7a-09e4d78948d3'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:47.768695 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-89ff5bc5-282b-47d9-8a7a-09e4d78948d3 admin admin] The token cache is being invalidate because project 1cd8850e224c434cbb7368083e9829a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:12:47.768981 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-89ff5bc5-282b-47d9-8a7a-09e4d78948d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:47.769642 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-89ff5bc5-282b-47d9-8a7a-09e4d78948d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:47.774148 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1139/2281] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:12:47 2022] DELETE /identity/v3/projects/1cd8850e224c434cbb7368083e9829a3 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:12:47.775450 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9d5c39-3276-44cf-a9d5-23a1ed2055c7 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:47.777085 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-fd9d5c39-3276-44cf-a9d5-23a1ed2055c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:12:47.790984 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9d5c39-3276-44cf-a9d5-23a1ed2055c7 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:47.863661 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4623c9-cc93-4bb7-8c82-3106bb3051f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:47.864547 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4623c9-cc93-4bb7-8c82-3106bb3051f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:47.864827 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4623c9-cc93-4bb7-8c82-3106bb3051f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:47.866030 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4623c9-cc93-4bb7-8c82-3106bb3051f3 admin admin] PATH_INFO: `/v3/users/ace9b1ecc017421285a0653dd9f4d2db` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:47.869555 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4623c9-cc93-4bb7-8c82-3106bb3051f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ace9b1ecc017421285a0653dd9f4d2db)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:47.871533 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4623c9-cc93-4bb7-8c82-3106bb3051f3 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:48.034568 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:12:48.191611 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-3e4623c9-cc93-4bb7-8c82-3106bb3051f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ace9b1ecc017421285a0653dd9f4d2db'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:48.215292 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-3e4623c9-cc93-4bb7-8c82-3106bb3051f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ace9b1ecc017421285a0653dd9f4d2db'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:48.224064 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-3e4623c9-cc93-4bb7-8c82-3106bb3051f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ace9b1ecc017421285a0653dd9f4d2db'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:48.224775 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-3e4623c9-cc93-4bb7-8c82-3106bb3051f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ace9b1ecc017421285a0653dd9f4d2db'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:48.225920 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-3e4623c9-cc93-4bb7-8c82-3106bb3051f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ace9b1ecc017421285a0653dd9f4d2db'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:48.343464 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1143/2282] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:12:47 2022] DELETE /identity/v3/users/ace9b1ecc017421285a0653dd9f4d2db => generated 0 bytes in 569 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:12:49.331801 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ff5bc5-282b-47d9-8a7a-09e4d78948d3 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:49.333647 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-89ff5bc5-282b-47d9-8a7a-09e4d78948d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:12:49.348543 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ff5bc5-282b-47d9-8a7a-09e4d78948d3 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:49.435236 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:12:49.445167 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fc89a5-46cc-4398-b4cf-cb0abfe48344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:49.446909 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fc89a5-46cc-4398-b4cf-cb0abfe48344 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:49.447264 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fc89a5-46cc-4398-b4cf-cb0abfe48344 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:49.447614 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fc89a5-46cc-4398-b4cf-cb0abfe48344 admin admin] PATH_INFO: `/v3/projects/7d3f60a3f7474a1ab0e3cc1f2d18e7e3` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:49.449078 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fc89a5-46cc-4398-b4cf-cb0abfe48344 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d3f60a3f7474a1ab0e3cc1f2d18e7e3)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:49.451479 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fc89a5-46cc-4398-b4cf-cb0abfe48344 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:49.593222 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-53fc89a5-46cc-4398-b4cf-cb0abfe48344 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d3f60a3f7474a1ab0e3cc1f2d18e7e3', 'request_id': 'req-53fc89a5-46cc-4398-b4cf-cb0abfe48344'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:49.735957 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-53fc89a5-46cc-4398-b4cf-cb0abfe48344 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d3f60a3f7474a1ab0e3cc1f2d18e7e3', 'request_id': 'req-53fc89a5-46cc-4398-b4cf-cb0abfe48344'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:49.745210 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-53fc89a5-46cc-4398-b4cf-cb0abfe48344 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d3f60a3f7474a1ab0e3cc1f2d18e7e3', 'request_id': 'req-53fc89a5-46cc-4398-b4cf-cb0abfe48344'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:49.759506 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-53fc89a5-46cc-4398-b4cf-cb0abfe48344 admin admin] The token cache is being invalidate because project 7d3f60a3f7474a1ab0e3cc1f2d18e7e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:12:49.759506 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-53fc89a5-46cc-4398-b4cf-cb0abfe48344 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:49.763234 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-53fc89a5-46cc-4398-b4cf-cb0abfe48344 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:12:49.765059 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1140/2283] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:12:49 2022] DELETE /identity/v3/projects/7d3f60a3f7474a1ab0e3cc1f2d18e7e3 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:12:49.770905 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b26dcb6-9e66-47f8-ac2a-6a56f7357d11 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:49.771684 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b26dcb6-9e66-47f8-ac2a-6a56f7357d11 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:49.772340 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b26dcb6-9e66-47f8-ac2a-6a56f7357d11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:49.792895 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-8b26dcb6-9e66-47f8-ac2a-6a56f7357d11 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:12:49.794368 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-8b26dcb6-9e66-47f8-ac2a-6a56f7357d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:12:49.856543 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1144/2284] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:12:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:12:49.864301 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fa4c9e-4161-4e0b-9a55-0e077eaee38d None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:49.864674 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fa4c9e-4161-4e0b-9a55-0e077eaee38d None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:49.864953 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fa4c9e-4161-4e0b-9a55-0e077eaee38d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:49.886761 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-70fa4c9e-4161-4e0b-9a55-0e077eaee38d None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:12:49.888096 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-70fa4c9e-4161-4e0b-9a55-0e077eaee38d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:12:49.896080 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1141/2285] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:12:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:12:49.901410 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b26dcb6-9e66-47f8-ac2a-6a56f7357d11 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:49.912009 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b26dcb6-9e66-47f8-ac2a-6a56f7357d11 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:49.920056 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ad13a5-832c-4cfb-8ccb-da3ef5b37a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:49.920056 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad13a5-832c-4cfb-8ccb-da3ef5b37a48 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:49.920056 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad13a5-832c-4cfb-8ccb-da3ef5b37a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:49.920056 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad13a5-832c-4cfb-8ccb-da3ef5b37a48 admin admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:49.920423 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ad13a5-832c-4cfb-8ccb-da3ef5b37a48 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 22:12:49.920423 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ad13a5-832c-4cfb-8ccb-da3ef5b37a48 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:49.922475 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ad13a5-832c-4cfb-8ccb-da3ef5b37a48 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:49.938099 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1145/2286] 10.222.0.18 () {58 vars in 1153 bytes} [Sat Aug 6 22:12:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 22:12:49.942230 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fa4c9e-4161-4e0b-9a55-0e077eaee38d None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:49.952443 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fa4c9e-4161-4e0b-9a55-0e077eaee38d None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:49.957443 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012a7260-5cf7-4b0b-b2b8-e7772a68ef37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:49.958047 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012a7260-5cf7-4b0b-b2b8-e7772a68ef37 admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:49.958326 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012a7260-5cf7-4b0b-b2b8-e7772a68ef37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:49.958594 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012a7260-5cf7-4b0b-b2b8-e7772a68ef37 admin admin] PATH_INFO: `/v3/projects` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:49.959672 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012a7260-5cf7-4b0b-b2b8-e7772a68ef37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:49.962129 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012a7260-5cf7-4b0b-b2b8-e7772a68ef37 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:50.183186 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1142/2287] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:12:49 2022] POST /identity/v3/projects => generated 391 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:12:50.187816 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ad13a5-832c-4cfb-8ccb-da3ef5b37a48 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:50.205386 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ad13a5-832c-4cfb-8ccb-da3ef5b37a48 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:50.300812 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594ec15e-363a-4656-9534-7423af077ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:50.301779 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ec15e-363a-4656-9534-7423af077ba5 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:50.302201 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ec15e-363a-4656-9534-7423af077ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:50.311756 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ec15e-363a-4656-9534-7423af077ba5 admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:50.311756 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594ec15e-363a-4656-9534-7423af077ba5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:50.311756 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594ec15e-363a-4656-9534-7423af077ba5 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:50.482763 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1146/2288] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:12:50 2022] POST /identity/v3/users => generated 296 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:12:50.484200 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012a7260-5cf7-4b0b-b2b8-e7772a68ef37 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:50.508657 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012a7260-5cf7-4b0b-b2b8-e7772a68ef37 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:50.514597 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874ba0d7-3646-4bd2-817b-da9a05f767c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:50.515236 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ba0d7-3646-4bd2-817b-da9a05f767c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:50.516001 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ba0d7-3646-4bd2-817b-da9a05f767c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:50.516496 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ba0d7-3646-4bd2-817b-da9a05f767c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:50.517324 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874ba0d7-3646-4bd2-817b-da9a05f767c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:50.519451 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874ba0d7-3646-4bd2-817b-da9a05f767c5 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:50.534176 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1143/2289] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:12:50 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:12:50.543151 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594ec15e-363a-4656-9534-7423af077ba5 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:50.558866 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594ec15e-363a-4656-9534-7423af077ba5 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:50.565693 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d809cae-a5e3-4868-b47f-9be216340d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:50.566819 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d809cae-a5e3-4868-b47f-9be216340d96 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:50.567257 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d809cae-a5e3-4868-b47f-9be216340d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:50.567730 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d809cae-a5e3-4868-b47f-9be216340d96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:50.568636 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d809cae-a5e3-4868-b47f-9be216340d96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:50.570533 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d809cae-a5e3-4868-b47f-9be216340d96 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:50.582897 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1147/2290] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:12:50 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:12:50.595416 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874ba0d7-3646-4bd2-817b-da9a05f767c5 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:50.606824 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874ba0d7-3646-4bd2-817b-da9a05f767c5 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:50.611847 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bbd7f6-9f7b-44b1-b137-1e1277b49297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:50.612588 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bbd7f6-9f7b-44b1-b137-1e1277b49297 admin admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:50.612865 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bbd7f6-9f7b-44b1-b137-1e1277b49297 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:50.613143 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bbd7f6-9f7b-44b1-b137-1e1277b49297 admin admin] PATH_INFO: `/v3/projects/1633d825dbe94630ad9e493a41a0d3e2/users/c29f703cb1a04506bca7a649edb27901/roles/56e4a96094e9434cbbafb22579c034f5` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:50.641368 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bbd7f6-9f7b-44b1-b137-1e1277b49297 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1633d825dbe94630ad9e493a41a0d3e2, user_id=c29f703cb1a04506bca7a649edb27901, role_id=56e4a96094e9434cbbafb22579c034f5)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:50.645862 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bbd7f6-9f7b-44b1-b137-1e1277b49297 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:50.778165 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1144/2291] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:12:50 2022] PUT /identity/v3/projects/1633d825dbe94630ad9e493a41a0d3e2/users/c29f703cb1a04506bca7a649edb27901/roles/56e4a96094e9434cbbafb22579c034f5 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:12:50.783097 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5974c8-c2af-4c14-9ed3-b99af5c71146 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:50.783525 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5974c8-c2af-4c14-9ed3-b99af5c71146 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:50.783624 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5974c8-c2af-4c14-9ed3-b99af5c71146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:50.821877 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-0d5974c8-c2af-4c14-9ed3-b99af5c71146 None None] MFA Rules not processed for user `c29f703cb1a04506bca7a649edb27901`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:12:50.823289 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-0d5974c8-c2af-4c14-9ed3-b99af5c71146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:12:50.896391 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1148/2292] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:12:50 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:12:51.013880 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bbd7f6-9f7b-44b1-b137-1e1277b49297 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:12:51.015837 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-50bbd7f6-9f7b-44b1-b137-1e1277b49297 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:12:51.076758 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bbd7f6-9f7b-44b1-b137-1e1277b49297 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:12:51.080725 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a2fa1b-2d80-44e0-a210-7749fc28059c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa71c71de263478da234f3c5716a6e5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:12:51.081336 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a2fa1b-2d80-44e0-a210-7749fc28059c service cinder] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:12:51.081712 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a2fa1b-2d80-44e0-a210-7749fc28059c service cinder] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:12:51.085198 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a2fa1b-2d80-44e0-a210-7749fc28059c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:12:51.091720 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a2fa1b-2d80-44e0-a210-7749fc28059c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:12:51.093194 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a2fa1b-2d80-44e0-a210-7749fc28059c service cinder] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:12:51.104814 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1145/2293] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:12:51 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:12:52.204080 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:13:03.015670 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=79,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:13:04.072941 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=76,keystone:INSERT=3 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:13:07.387578 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34bee45-9cb1-4e0a-b73f-6fc051c252a8 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:13:07.388099 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34bee45-9cb1-4e0a-b73f-6fc051c252a8 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:13:07.388099 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34bee45-9cb1-4e0a-b73f-6fc051c252a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:13:07.411756 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-e34bee45-9cb1-4e0a-b73f-6fc051c252a8 None None] MFA Rules not processed for user `d1b26ba592724dd389584bb800d877a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:13:07.411756 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-e34bee45-9cb1-4e0a-b73f-6fc051c252a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:13:07.481782 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1149/2294] 10.222.0.18 () {58 vars in 1009 bytes} [Sat Aug 6 22:13:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:13:07.532156 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a2fa1b-2d80-44e0-a210-7749fc28059c service cinder] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:13:07.533645 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-b6a2fa1b-2d80-44e0-a210-7749fc28059c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:13:07.616380 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a2fa1b-2d80-44e0-a210-7749fc28059c service cinder] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:13:07.622237 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6f0f89-7881-4d5d-8f28-55183358a12e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7fc9da7505744608dfd3bbd46a0a9d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:13:07.626939 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6f0f89-7881-4d5d-8f28-55183358a12e service swift] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:13:07.626939 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6f0f89-7881-4d5d-8f28-55183358a12e service swift] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:13:07.626939 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6f0f89-7881-4d5d-8f28-55183358a12e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:13:07.631854 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6f0f89-7881-4d5d-8f28-55183358a12e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:13:07.633447 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6f0f89-7881-4d5d-8f28-55183358a12e service swift] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:13:07.642062 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1146/2295] 10.222.0.18 () {60 vars in 1384 bytes} [Sat Aug 6 22:13:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 22:13:17.472491 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=25 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:13:17.639013 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=32 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:13:30.048521 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea65d1-e666-43f7-b674-bee0f5f591db None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:13:30.049684 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea65d1-e666-43f7-b674-bee0f5f591db None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:13:30.049923 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea65d1-e666-43f7-b674-bee0f5f591db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:13:30.069746 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-65ea65d1-e666-43f7-b674-bee0f5f591db None None] MFA Rules not processed for user `d1b26ba592724dd389584bb800d877a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:13:30.071207 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-65ea65d1-e666-43f7-b674-bee0f5f591db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:13:30.078714 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1150/2296] 10.222.0.18 () {58 vars in 967 bytes} [Sat Aug 6 22:13:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:13:30.092244 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6f0f89-7881-4d5d-8f28-55183358a12e service swift] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:13:30.104889 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6f0f89-7881-4d5d-8f28-55183358a12e service swift] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:13:30.110089 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16980f14-698b-497d-a030-6f6141113bb5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7fc9da7505744608dfd3bbd46a0a9d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:13:30.110731 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16980f14-698b-497d-a030-6f6141113bb5 service swift] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:13:30.111007 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16980f14-698b-497d-a030-6f6141113bb5 service swift] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:13:30.111274 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16980f14-698b-497d-a030-6f6141113bb5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:13:30.122640 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16980f14-698b-497d-a030-6f6141113bb5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:13:30.124712 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16980f14-698b-497d-a030-6f6141113bb5 service swift] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:13:30.134627 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1147/2297] 10.222.0.18 () {60 vars in 1384 bytes} [Sat Aug 6 22:13:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 22:13:35.408813 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ea65d1-e666-43f7-b674-bee0f5f591db None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:13:35.416148 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-65ea65d1-e666-43f7-b674-bee0f5f591db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:13:35.481386 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ea65d1-e666-43f7-b674-bee0f5f591db None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:13:35.487214 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3668a7cc-b341-4492-9957-bd1713a3c32b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '680d62a67ae2456f8c129e26728c3297', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:13:35.487214 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3668a7cc-b341-4492-9957-bd1713a3c32b service placement] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:13:35.487507 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3668a7cc-b341-4492-9957-bd1713a3c32b service placement] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:13:35.487795 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3668a7cc-b341-4492-9957-bd1713a3c32b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:13:35.489598 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-3668a7cc-b341-4492-9957-bd1713a3c32b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:13:35.502173 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3668a7cc-b341-4492-9957-bd1713a3c32b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:13:35.503806 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3668a7cc-b341-4492-9957-bd1713a3c32b service placement] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:13:35.515055 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1151/2298] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:13:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:13:37.333818 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16980f14-698b-497d-a030-6f6141113bb5 service swift] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:13:37.335350 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-16980f14-698b-497d-a030-6f6141113bb5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:13:37.426286 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16980f14-698b-497d-a030-6f6141113bb5 service swift] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:13:37.431035 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad2b287-00d5-4c35-9c5a-d1f2010d9b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:13:37.431758 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad2b287-00d5-4c35-9c5a-d1f2010d9b36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:13:37.432097 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad2b287-00d5-4c35-9c5a-d1f2010d9b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:13:37.432451 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad2b287-00d5-4c35-9c5a-d1f2010d9b36 admin admin] PATH_INFO: `/v3/users/26b6e4a0640548dd8d6bb800d1f26b95` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:13:37.433665 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad2b287-00d5-4c35-9c5a-d1f2010d9b36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26b6e4a0640548dd8d6bb800d1f26b95)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:13:37.435639 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad2b287-00d5-4c35-9c5a-d1f2010d9b36 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:13:37.645557 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-bad2b287-00d5-4c35-9c5a-d1f2010d9b36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26b6e4a0640548dd8d6bb800d1f26b95'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:13:37.660668 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-bad2b287-00d5-4c35-9c5a-d1f2010d9b36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26b6e4a0640548dd8d6bb800d1f26b95'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:13:37.739098 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-bad2b287-00d5-4c35-9c5a-d1f2010d9b36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26b6e4a0640548dd8d6bb800d1f26b95'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:13:37.749523 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-bad2b287-00d5-4c35-9c5a-d1f2010d9b36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26b6e4a0640548dd8d6bb800d1f26b95'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:13:37.750244 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-bad2b287-00d5-4c35-9c5a-d1f2010d9b36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26b6e4a0640548dd8d6bb800d1f26b95'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:13:37.757145 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1148/2299] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:13:37 2022] DELETE /identity/v3/users/26b6e4a0640548dd8d6bb800d1f26b95 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:13:38.026927 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3668a7cc-b341-4492-9957-bd1713a3c32b service placement] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:13:38.042208 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3668a7cc-b341-4492-9957-bd1713a3c32b service placement] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:13:38.107879 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1e7c97-9a8a-46a2-9d6c-418877939fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:13:38.108546 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1e7c97-9a8a-46a2-9d6c-418877939fb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:13:38.109022 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1e7c97-9a8a-46a2-9d6c-418877939fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:13:38.109239 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1e7c97-9a8a-46a2-9d6c-418877939fb6 admin admin] PATH_INFO: `/v3/projects/6cc6b84f1fcc4e28b7c3a7547b6640b8` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:13:38.110387 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1e7c97-9a8a-46a2-9d6c-418877939fb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cc6b84f1fcc4e28b7c3a7547b6640b8)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:13:38.112312 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1e7c97-9a8a-46a2-9d6c-418877939fb6 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:13:38.233118 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-0c1e7c97-9a8a-46a2-9d6c-418877939fb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cc6b84f1fcc4e28b7c3a7547b6640b8', 'request_id': 'req-0c1e7c97-9a8a-46a2-9d6c-418877939fb6'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:13:38.305306 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-0c1e7c97-9a8a-46a2-9d6c-418877939fb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cc6b84f1fcc4e28b7c3a7547b6640b8', 'request_id': 'req-0c1e7c97-9a8a-46a2-9d6c-418877939fb6'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:13:38.314961 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-0c1e7c97-9a8a-46a2-9d6c-418877939fb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cc6b84f1fcc4e28b7c3a7547b6640b8', 'request_id': 'req-0c1e7c97-9a8a-46a2-9d6c-418877939fb6'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:13:38.328000 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-0c1e7c97-9a8a-46a2-9d6c-418877939fb6 admin admin] The token cache is being invalidate because project 6cc6b84f1fcc4e28b7c3a7547b6640b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:13:38.328371 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-0c1e7c97-9a8a-46a2-9d6c-418877939fb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:13:38.329064 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-0c1e7c97-9a8a-46a2-9d6c-418877939fb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:13:38.330907 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1152/2300] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:13:38 2022] DELETE /identity/v3/projects/6cc6b84f1fcc4e28b7c3a7547b6640b8 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:13:38.335951 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e14b05-cf93-41c3-8eb8-180cbb9bfb92 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:13:38.336632 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e14b05-cf93-41c3-8eb8-180cbb9bfb92 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:13:38.336632 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e14b05-cf93-41c3-8eb8-180cbb9bfb92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:13:38.362679 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-85e14b05-cf93-41c3-8eb8-180cbb9bfb92 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:13:38.364694 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-85e14b05-cf93-41c3-8eb8-180cbb9bfb92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:13:38.426332 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1149/2301] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:13:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:13:38.434538 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0fbc9-3ee7-4762-bc72-c2e0b35ec6f0 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:13:38.434835 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0fbc9-3ee7-4762-bc72-c2e0b35ec6f0 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:13:38.435108 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0fbc9-3ee7-4762-bc72-c2e0b35ec6f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:13:38.456740 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-5fd0fbc9-3ee7-4762-bc72-c2e0b35ec6f0 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:13:38.457949 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-5fd0fbc9-3ee7-4762-bc72-c2e0b35ec6f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:13:38.467290 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1153/2302] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:13:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:13:38.471624 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e14b05-cf93-41c3-8eb8-180cbb9bfb92 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:13:38.482728 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e14b05-cf93-41c3-8eb8-180cbb9bfb92 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:13:38.487886 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965fbe3e-26d1-43ae-8143-354320edf4b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:13:38.488698 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965fbe3e-26d1-43ae-8143-354320edf4b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:13:38.489229 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965fbe3e-26d1-43ae-8143-354320edf4b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:13:38.489508 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965fbe3e-26d1-43ae-8143-354320edf4b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:13:38.490165 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965fbe3e-26d1-43ae-8143-354320edf4b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 22:13:38.490899 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965fbe3e-26d1-43ae-8143-354320edf4b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:13:38.492800 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965fbe3e-26d1-43ae-8143-354320edf4b6 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:13:38.494438 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:13:38.505021 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1150/2303] 10.222.0.18 () {58 vars in 1153 bytes} [Sat Aug 6 22:13:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 22:13:38.511450 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd0fbc9-3ee7-4762-bc72-c2e0b35ec6f0 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:13:38.520756 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd0fbc9-3ee7-4762-bc72-c2e0b35ec6f0 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:13:38.525715 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b51e61-e651-425c-9789-223e6b8bb9bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:13:38.526311 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b51e61-e651-425c-9789-223e6b8bb9bd admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:13:38.526590 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b51e61-e651-425c-9789-223e6b8bb9bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:13:38.526864 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b51e61-e651-425c-9789-223e6b8bb9bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:13:38.527485 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b51e61-e651-425c-9789-223e6b8bb9bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:13:38.529425 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b51e61-e651-425c-9789-223e6b8bb9bd admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:13:38.533892 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:13:38.687172 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1154/2304] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:13:38 2022] POST /identity/v3/projects => generated 369 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:13:38.691345 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965fbe3e-26d1-43ae-8143-354320edf4b6 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:13:38.702142 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965fbe3e-26d1-43ae-8143-354320edf4b6 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:13:38.771905 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801e8f70-8e9d-4ef4-bf02-2689780ba29c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:13:38.772710 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801e8f70-8e9d-4ef4-bf02-2689780ba29c admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:13:38.773104 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801e8f70-8e9d-4ef4-bf02-2689780ba29c admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:13:38.773426 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801e8f70-8e9d-4ef4-bf02-2689780ba29c admin admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:13:38.774202 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801e8f70-8e9d-4ef4-bf02-2689780ba29c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:13:38.776295 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801e8f70-8e9d-4ef4-bf02-2689780ba29c admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:13:38.893587 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1151/2305] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:13:38 2022] POST /identity/v3/users => generated 285 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:13:38.897435 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b51e61-e651-425c-9789-223e6b8bb9bd admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:13:38.908497 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b51e61-e651-425c-9789-223e6b8bb9bd admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:13:38.913353 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e895f54c-2fb6-42c1-95ae-e34a4b0ff7c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:13:38.914531 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e895f54c-2fb6-42c1-95ae-e34a4b0ff7c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:13:38.914810 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e895f54c-2fb6-42c1-95ae-e34a4b0ff7c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:13:38.915080 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e895f54c-2fb6-42c1-95ae-e34a4b0ff7c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:13:38.915716 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e895f54c-2fb6-42c1-95ae-e34a4b0ff7c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:13:38.917361 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e895f54c-2fb6-42c1-95ae-e34a4b0ff7c6 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:13:38.927326 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1155/2306] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:13:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:13:38.931538 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801e8f70-8e9d-4ef4-bf02-2689780ba29c admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:13:38.941565 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801e8f70-8e9d-4ef4-bf02-2689780ba29c admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:13:38.946602 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc2c478-4e31-4827-a49e-26042b4a6f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:13:38.947206 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc2c478-4e31-4827-a49e-26042b4a6f68 admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:13:38.947488 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc2c478-4e31-4827-a49e-26042b4a6f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:13:38.947782 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc2c478-4e31-4827-a49e-26042b4a6f68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:13:38.948448 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc2c478-4e31-4827-a49e-26042b4a6f68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:13:38.950049 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc2c478-4e31-4827-a49e-26042b4a6f68 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:13:38.963693 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1152/2307] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:13:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:13:38.965253 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e895f54c-2fb6-42c1-95ae-e34a4b0ff7c6 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:13:38.975660 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e895f54c-2fb6-42c1-95ae-e34a4b0ff7c6 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:13:38.980846 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be399591-0520-408c-84a6-bcbeb9613e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:13:38.981496 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be399591-0520-408c-84a6-bcbeb9613e99 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:13:38.981776 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be399591-0520-408c-84a6-bcbeb9613e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:13:38.982048 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be399591-0520-408c-84a6-bcbeb9613e99 admin admin] PATH_INFO: `/v3/projects/43c7e816ab684850a033371d64b8c0c9/users/11db88f3a6ab48f5a2b078d42628d7c3/roles/56e4a96094e9434cbbafb22579c034f5` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:13:38.996250 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be399591-0520-408c-84a6-bcbeb9613e99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43c7e816ab684850a033371d64b8c0c9, user_id=11db88f3a6ab48f5a2b078d42628d7c3, role_id=56e4a96094e9434cbbafb22579c034f5)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:13:38.998947 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be399591-0520-408c-84a6-bcbeb9613e99 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:13:39.047170 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1156/2308] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:13:38 2022] PUT /identity/v3/projects/43c7e816ab684850a033371d64b8c0c9/users/11db88f3a6ab48f5a2b078d42628d7c3/roles/56e4a96094e9434cbbafb22579c034f5 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:13:39.058240 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeca287-67cd-4f97-a74c-e8197faec388 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:13:39.058517 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeca287-67cd-4f97-a74c-e8197faec388 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:13:39.058787 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeca287-67cd-4f97-a74c-e8197faec388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:13:39.086217 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-3eeca287-67cd-4f97-a74c-e8197faec388 None None] MFA Rules not processed for user `11db88f3a6ab48f5a2b078d42628d7c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:13:39.087464 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-3eeca287-67cd-4f97-a74c-e8197faec388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:13:39.150825 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1153/2309] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:13:39 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:13:39.162388 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be399591-0520-408c-84a6-bcbeb9613e99 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:13:39.164908 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-be399591-0520-408c-84a6-bcbeb9613e99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:13:39.220794 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be399591-0520-408c-84a6-bcbeb9613e99 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:13:39.224355 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dc0437-9b81-4c06-af46-16bd64a2a9dc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8d38e73e105476e8de858cfa939ba20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:13:39.225234 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dc0437-9b81-4c06-af46-16bd64a2a9dc service glance] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:13:39.225518 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dc0437-9b81-4c06-af46-16bd64a2a9dc service glance] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:13:39.225792 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dc0437-9b81-4c06-af46-16bd64a2a9dc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:13:39.234731 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dc0437-9b81-4c06-af46-16bd64a2a9dc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:13:39.236103 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dc0437-9b81-4c06-af46-16bd64a2a9dc service glance] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:13:39.245195 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1157/2310] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:13:39 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:13:43.729268 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeca287-67cd-4f97-a74c-e8197faec388 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:13:43.730750 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-3eeca287-67cd-4f97-a74c-e8197faec388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:13:43.798134 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeca287-67cd-4f97-a74c-e8197faec388 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:13:43.802664 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8f07ca-ce27-44d6-8760-fe8b6cc097d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:13:43.803288 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8f07ca-ce27-44d6-8760-fe8b6cc097d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:13:43.803561 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8f07ca-ce27-44d6-8760-fe8b6cc097d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:13:43.803827 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8f07ca-ce27-44d6-8760-fe8b6cc097d7 admin admin] PATH_INFO: `/v3/users/c29f703cb1a04506bca7a649edb27901` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:13:43.804963 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8f07ca-ce27-44d6-8760-fe8b6cc097d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c29f703cb1a04506bca7a649edb27901)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:13:43.807258 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8f07ca-ce27-44d6-8760-fe8b6cc097d7 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:13:43.809598 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:13:44.020342 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-0d8f07ca-ce27-44d6-8760-fe8b6cc097d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c29f703cb1a04506bca7a649edb27901'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:13:44.034208 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-0d8f07ca-ce27-44d6-8760-fe8b6cc097d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c29f703cb1a04506bca7a649edb27901'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:13:44.106628 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-0d8f07ca-ce27-44d6-8760-fe8b6cc097d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c29f703cb1a04506bca7a649edb27901'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:13:44.116040 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-0d8f07ca-ce27-44d6-8760-fe8b6cc097d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c29f703cb1a04506bca7a649edb27901'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:13:44.116763 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-0d8f07ca-ce27-44d6-8760-fe8b6cc097d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c29f703cb1a04506bca7a649edb27901'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:13:44.123094 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1154/2311] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:13:43 2022] DELETE /identity/v3/users/c29f703cb1a04506bca7a649edb27901 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:13:44.127565 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7738220f-71e0-4426-84b4-212e6df45d0e None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:13:44.128059 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7738220f-71e0-4426-84b4-212e6df45d0e None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:13:44.128156 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7738220f-71e0-4426-84b4-212e6df45d0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:13:44.148387 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-7738220f-71e0-4426-84b4-212e6df45d0e None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:13:44.150374 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-7738220f-71e0-4426-84b4-212e6df45d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:13:44.219800 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1158/2312] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:13:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:13:44.230327 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8f07ca-ce27-44d6-8760-fe8b6cc097d7 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:13:44.231889 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-0d8f07ca-ce27-44d6-8760-fe8b6cc097d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:13:44.300677 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8f07ca-ce27-44d6-8760-fe8b6cc097d7 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:13:44.304324 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ab7bed-c391-4599-a7d2-5757ff44a1be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:13:44.304958 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ab7bed-c391-4599-a7d2-5757ff44a1be service neutron] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:13:44.305236 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ab7bed-c391-4599-a7d2-5757ff44a1be service neutron] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:13:44.305507 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ab7bed-c391-4599-a7d2-5757ff44a1be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:13:44.314572 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ab7bed-c391-4599-a7d2-5757ff44a1be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:13:44.315981 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ab7bed-c391-4599-a7d2-5757ff44a1be service neutron] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:13:44.326839 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1155/2313] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:13:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:13:44.744582 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7738220f-71e0-4426-84b4-212e6df45d0e None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:13:44.745990 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-7738220f-71e0-4426-84b4-212e6df45d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:13:44.762286 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7738220f-71e0-4426-84b4-212e6df45d0e None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:13:44.768513 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcdb2bd-f237-4708-86e0-0ebf2d2c8605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:13:44.769153 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcdb2bd-f237-4708-86e0-0ebf2d2c8605 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:13:44.769464 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcdb2bd-f237-4708-86e0-0ebf2d2c8605 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:13:44.769700 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcdb2bd-f237-4708-86e0-0ebf2d2c8605 admin admin] PATH_INFO: `/v3/projects/1633d825dbe94630ad9e493a41a0d3e2` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:13:44.770832 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcdb2bd-f237-4708-86e0-0ebf2d2c8605 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1633d825dbe94630ad9e493a41a0d3e2)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:13:44.772841 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcdb2bd-f237-4708-86e0-0ebf2d2c8605 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:13:44.898579 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:13:44.899938 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-1fcdb2bd-f237-4708-86e0-0ebf2d2c8605 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1633d825dbe94630ad9e493a41a0d3e2', 'request_id': 'req-1fcdb2bd-f237-4708-86e0-0ebf2d2c8605'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:13:44.961422 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-1fcdb2bd-f237-4708-86e0-0ebf2d2c8605 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1633d825dbe94630ad9e493a41a0d3e2', 'request_id': 'req-1fcdb2bd-f237-4708-86e0-0ebf2d2c8605'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:13:44.967285 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-1fcdb2bd-f237-4708-86e0-0ebf2d2c8605 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1633d825dbe94630ad9e493a41a0d3e2', 'request_id': 'req-1fcdb2bd-f237-4708-86e0-0ebf2d2c8605'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:13:44.979786 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-1fcdb2bd-f237-4708-86e0-0ebf2d2c8605 admin admin] The token cache is being invalidate because project 1633d825dbe94630ad9e493a41a0d3e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:13:44.980081 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-1fcdb2bd-f237-4708-86e0-0ebf2d2c8605 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:13:44.980724 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-1fcdb2bd-f237-4708-86e0-0ebf2d2c8605 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:13:44.982239 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1159/2314] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:13:44 2022] DELETE /identity/v3/projects/1633d825dbe94630ad9e493a41a0d3e2 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:13:44.989416 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f89150-f411-41ba-8b36-77c41a9d5fbb None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:13:44.989921 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f89150-f411-41ba-8b36-77c41a9d5fbb None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:13:44.989921 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f89150-f411-41ba-8b36-77c41a9d5fbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:13:45.010091 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-85f89150-f411-41ba-8b36-77c41a9d5fbb None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:13:45.011324 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-85f89150-f411-41ba-8b36-77c41a9d5fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:13:45.071111 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1156/2315] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:13:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:13:45.079464 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa43801-0014-4df6-bec1-0c0478562356 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:13:45.079851 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa43801-0014-4df6-bec1-0c0478562356 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:13:45.080131 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa43801-0014-4df6-bec1-0c0478562356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:13:45.100879 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-0fa43801-0014-4df6-bec1-0c0478562356 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:13:45.102084 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-0fa43801-0014-4df6-bec1-0c0478562356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:13:45.110611 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1160/2316] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:13:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 22:13:45.114833 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f89150-f411-41ba-8b36-77c41a9d5fbb None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:13:45.125651 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f89150-f411-41ba-8b36-77c41a9d5fbb None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:13:45.130323 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d29a12-4c06-4142-bb7c-5068b874da3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:13:45.130920 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d29a12-4c06-4142-bb7c-5068b874da3b admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:13:45.131198 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d29a12-4c06-4142-bb7c-5068b874da3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:13:45.131470 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d29a12-4c06-4142-bb7c-5068b874da3b admin admin] PATH_INFO: `/v3/domains` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:13:45.132116 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d29a12-4c06-4142-bb7c-5068b874da3b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 22:13:45.132437 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d29a12-4c06-4142-bb7c-5068b874da3b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:13:45.134083 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d29a12-4c06-4142-bb7c-5068b874da3b admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:13:45.146247 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1157/2317] 10.222.0.18 () {58 vars in 1153 bytes} [Sat Aug 6 22:13:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 22:13:45.156979 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa43801-0014-4df6-bec1-0c0478562356 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:13:45.168959 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa43801-0014-4df6-bec1-0c0478562356 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:13:45.174360 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edc7396-4f20-4eb0-9f29-80c1d5d57609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:13:45.174978 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edc7396-4f20-4eb0-9f29-80c1d5d57609 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:13:45.175255 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edc7396-4f20-4eb0-9f29-80c1d5d57609 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:13:45.175545 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edc7396-4f20-4eb0-9f29-80c1d5d57609 admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:13:45.176197 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edc7396-4f20-4eb0-9f29-80c1d5d57609 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:13:45.178170 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edc7396-4f20-4eb0-9f29-80c1d5d57609 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:13:45.305728 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1161/2318] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:13:45 2022] POST /identity/v3/projects => generated 379 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:13:45.309886 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d29a12-4c06-4142-bb7c-5068b874da3b admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:13:45.320056 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d29a12-4c06-4142-bb7c-5068b874da3b admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:13:45.328570 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:13:45.395668 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3156f645-4c7b-44c7-b53e-a7574427f219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:13:45.396275 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3156f645-4c7b-44c7-b53e-a7574427f219 admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:13:45.396681 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3156f645-4c7b-44c7-b53e-a7574427f219 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:13:45.396968 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3156f645-4c7b-44c7-b53e-a7574427f219 admin admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:13:45.397595 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3156f645-4c7b-44c7-b53e-a7574427f219 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:13:45.400384 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3156f645-4c7b-44c7-b53e-a7574427f219 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:13:45.470544 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1158/2319] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:13:45 2022] POST /identity/v3/users => generated 290 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:13:45.474537 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edc7396-4f20-4eb0-9f29-80c1d5d57609 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:13:45.485732 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edc7396-4f20-4eb0-9f29-80c1d5d57609 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:13:45.490344 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a93ae7-6067-4daf-a99f-4f826ca9fe03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:13:45.490936 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a93ae7-6067-4daf-a99f-4f826ca9fe03 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:13:45.491210 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a93ae7-6067-4daf-a99f-4f826ca9fe03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:13:45.491478 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a93ae7-6067-4daf-a99f-4f826ca9fe03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:13:45.492115 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a93ae7-6067-4daf-a99f-4f826ca9fe03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:13:45.493832 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a93ae7-6067-4daf-a99f-4f826ca9fe03 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:13:45.504727 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1162/2320] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:13:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:13:45.509166 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3156f645-4c7b-44c7-b53e-a7574427f219 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:13:45.519785 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3156f645-4c7b-44c7-b53e-a7574427f219 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:13:45.525272 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a3e0d9-24c7-4577-aaf0-bbc82b5badca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:13:45.525906 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a3e0d9-24c7-4577-aaf0-bbc82b5badca admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:13:45.526184 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a3e0d9-24c7-4577-aaf0-bbc82b5badca admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:13:45.526454 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a3e0d9-24c7-4577-aaf0-bbc82b5badca admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:13:45.527099 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a3e0d9-24c7-4577-aaf0-bbc82b5badca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:13:45.528711 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a3e0d9-24c7-4577-aaf0-bbc82b5badca admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:13:45.538768 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1159/2321] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:13:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:13:45.543529 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a93ae7-6067-4daf-a99f-4f826ca9fe03 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:13:45.553732 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a93ae7-6067-4daf-a99f-4f826ca9fe03 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:13:45.558687 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c9a19f-1f1e-4e08-ae09-12717f86d02b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:13:45.559324 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c9a19f-1f1e-4e08-ae09-12717f86d02b admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:13:45.559761 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c9a19f-1f1e-4e08-ae09-12717f86d02b admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:13:45.559974 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c9a19f-1f1e-4e08-ae09-12717f86d02b admin admin] PATH_INFO: `/v3/projects/a375619d804c4675a5b9115aa13759a8/users/7a851cc02ef54ebc9101964bb5c0d9d7/roles/56e4a96094e9434cbbafb22579c034f5` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:13:45.575286 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c9a19f-1f1e-4e08-ae09-12717f86d02b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a375619d804c4675a5b9115aa13759a8, user_id=7a851cc02ef54ebc9101964bb5c0d9d7, role_id=56e4a96094e9434cbbafb22579c034f5)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:13:45.578184 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c9a19f-1f1e-4e08-ae09-12717f86d02b admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:13:45.648315 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1163/2322] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:13:45 2022] PUT /identity/v3/projects/a375619d804c4675a5b9115aa13759a8/users/7a851cc02ef54ebc9101964bb5c0d9d7/roles/56e4a96094e9434cbbafb22579c034f5 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:13:45.654059 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263f175b-f0c1-4a8b-ab03-66f61325102b None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:13:45.654339 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263f175b-f0c1-4a8b-ab03-66f61325102b None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:13:45.654631 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263f175b-f0c1-4a8b-ab03-66f61325102b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:13:45.673625 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-263f175b-f0c1-4a8b-ab03-66f61325102b None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:13:45.674932 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-263f175b-f0c1-4a8b-ab03-66f61325102b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:13:45.735023 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1160/2323] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:13:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:13:45.745485 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c9a19f-1f1e-4e08-ae09-12717f86d02b admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:13:45.747202 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-c3c9a19f-1f1e-4e08-ae09-12717f86d02b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:13:45.811491 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c9a19f-1f1e-4e08-ae09-12717f86d02b admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:13:45.815253 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486bce7d-30d4-4ce8-9221-453a44c0f127 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:13:45.815863 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486bce7d-30d4-4ce8-9221-453a44c0f127 service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:13:45.816140 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486bce7d-30d4-4ce8-9221-453a44c0f127 service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:13:45.816473 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486bce7d-30d4-4ce8-9221-453a44c0f127 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:13:45.828057 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486bce7d-30d4-4ce8-9221-453a44c0f127 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:13:45.829540 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486bce7d-30d4-4ce8-9221-453a44c0f127 service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:13:45.843636 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1164/2324] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:13:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 22:13:52.279589 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ff2e9c-e367-4fc7-875d-ef583e32aa31 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:13:52.280137 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ff2e9c-e367-4fc7-875d-ef583e32aa31 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:13:52.280137 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ff2e9c-e367-4fc7-875d-ef583e32aa31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:13:52.313269 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-95ff2e9c-e367-4fc7-875d-ef583e32aa31 None None] MFA Rules not processed for user `7a851cc02ef54ebc9101964bb5c0d9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:13:52.324267 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-95ff2e9c-e367-4fc7-875d-ef583e32aa31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:13:52.408353 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1161/2325] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:13:52 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:13:52.409511 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486bce7d-30d4-4ce8-9221-453a44c0f127 service neutron] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:13:52.420769 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486bce7d-30d4-4ce8-9221-453a44c0f127 service neutron] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:13:52.434835 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f2bc9a-2c00-471b-b325-93879d84792d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:13:52.435649 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f2bc9a-2c00-471b-b325-93879d84792d admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:13:52.435994 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f2bc9a-2c00-471b-b325-93879d84792d admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:13:52.436376 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f2bc9a-2c00-471b-b325-93879d84792d admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:13:52.437205 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f2bc9a-2c00-471b-b325-93879d84792d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:13:52.439246 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f2bc9a-2c00-471b-b325-93879d84792d admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:13:52.546822 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1165/2326] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:13:52 2022] POST /identity/v3/projects => generated 383 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 22:13:52.550980 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ff2e9c-e367-4fc7-875d-ef583e32aa31 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:13:52.562482 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ff2e9c-e367-4fc7-875d-ef583e32aa31 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:13:52.581393 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:13:52.627321 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e51db0-7522-4d11-aaf2-83ad70097914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:13:52.627970 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e51db0-7522-4d11-aaf2-83ad70097914 admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:13:52.628247 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e51db0-7522-4d11-aaf2-83ad70097914 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:13:52.628553 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e51db0-7522-4d11-aaf2-83ad70097914 admin admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:13:52.629219 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e51db0-7522-4d11-aaf2-83ad70097914 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:13:52.631127 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e51db0-7522-4d11-aaf2-83ad70097914 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:13:52.718618 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1162/2327] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:13:52 2022] POST /identity/v3/users => generated 298 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:13:52.719824 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f2bc9a-2c00-471b-b325-93879d84792d admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:13:52.731879 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f2bc9a-2c00-471b-b325-93879d84792d admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:13:52.739003 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d59a7b3-13cd-4371-bec9-6043f45c655c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:13:52.740128 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d59a7b3-13cd-4371-bec9-6043f45c655c admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:13:52.740546 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d59a7b3-13cd-4371-bec9-6043f45c655c admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:13:52.745253 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d59a7b3-13cd-4371-bec9-6043f45c655c admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:13:52.745253 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d59a7b3-13cd-4371-bec9-6043f45c655c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:13:52.745253 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d59a7b3-13cd-4371-bec9-6043f45c655c admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:13:52.758739 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1166/2328] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:13:52 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:13:52.762985 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e51db0-7522-4d11-aaf2-83ad70097914 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:13:52.778032 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e51db0-7522-4d11-aaf2-83ad70097914 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:13:52.783191 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c87a1c-3f93-4cdf-b357-00cd824c38c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:13:52.783842 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c87a1c-3f93-4cdf-b357-00cd824c38c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:13:52.784124 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c87a1c-3f93-4cdf-b357-00cd824c38c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:13:52.784441 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c87a1c-3f93-4cdf-b357-00cd824c38c0 admin admin] PATH_INFO: `/v3/projects/56add7eeecdc45ddb0f142a6ab36754c/users/d960b33daee341d98c3e41fb690adbb3/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:13:52.802638 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c87a1c-3f93-4cdf-b357-00cd824c38c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56add7eeecdc45ddb0f142a6ab36754c, user_id=d960b33daee341d98c3e41fb690adbb3, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:13:52.805993 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c87a1c-3f93-4cdf-b357-00cd824c38c0 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:13:52.898984 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1163/2329] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:13:52 2022] PUT /identity/v3/projects/56add7eeecdc45ddb0f142a6ab36754c/users/d960b33daee341d98c3e41fb690adbb3/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:13:52.903757 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d59a7b3-13cd-4371-bec9-6043f45c655c admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:13:52.913824 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d59a7b3-13cd-4371-bec9-6043f45c655c admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:13:52.957129 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:13:52.974514 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931de9bd-190e-448d-a131-e08bfe16f937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:13:52.975405 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931de9bd-190e-448d-a131-e08bfe16f937 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:13:52.975686 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931de9bd-190e-448d-a131-e08bfe16f937 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:13:52.975966 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931de9bd-190e-448d-a131-e08bfe16f937 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:13:52.976720 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931de9bd-190e-448d-a131-e08bfe16f937 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:13:52.978880 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931de9bd-190e-448d-a131-e08bfe16f937 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:13:52.996969 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1167/2330] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:13:52 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:13:52.998738 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c87a1c-3f93-4cdf-b357-00cd824c38c0 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:13:53.010974 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c87a1c-3f93-4cdf-b357-00cd824c38c0 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:13:53.016023 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fafaa6c-4353-4aa0-9718-5fcb91051d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:13:53.016741 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fafaa6c-4353-4aa0-9718-5fcb91051d95 admin admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:13:53.017019 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fafaa6c-4353-4aa0-9718-5fcb91051d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:13:53.017296 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fafaa6c-4353-4aa0-9718-5fcb91051d95 admin admin] PATH_INFO: `/v3/projects/56add7eeecdc45ddb0f142a6ab36754c/users/d960b33daee341d98c3e41fb690adbb3/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:13:53.019208 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fafaa6c-4353-4aa0-9718-5fcb91051d95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56add7eeecdc45ddb0f142a6ab36754c, user_id=d960b33daee341d98c3e41fb690adbb3, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:13:53.021927 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fafaa6c-4353-4aa0-9718-5fcb91051d95 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:13:53.033658 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1164/2331] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:13:52 2022] PUT /identity/v3/projects/56add7eeecdc45ddb0f142a6ab36754c/users/d960b33daee341d98c3e41fb690adbb3/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:13:59.376768 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ad0b94-d4e3-430c-b7c6-1834cb048283 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:13:59.377375 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ad0b94-d4e3-430c-b7c6-1834cb048283 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:13:59.377590 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ad0b94-d4e3-430c-b7c6-1834cb048283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:13:59.406998 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-79ad0b94-d4e3-430c-b7c6-1834cb048283 None None] MFA Rules not processed for user `d960b33daee341d98c3e41fb690adbb3`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:13:59.408245 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-79ad0b94-d4e3-430c-b7c6-1834cb048283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:13:59.481774 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1168/2332] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:13:59 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:13:59.494963 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fafaa6c-4353-4aa0-9718-5fcb91051d95 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:13:59.496202 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-0fafaa6c-4353-4aa0-9718-5fcb91051d95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:13:59.560316 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fafaa6c-4353-4aa0-9718-5fcb91051d95 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:13:59.563951 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0754ccd3-cf5e-48ba-9c87-8bc9b438cdd3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8d38e73e105476e8de858cfa939ba20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:13:59.564583 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0754ccd3-cf5e-48ba-9c87-8bc9b438cdd3 service glance] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:13:59.564857 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0754ccd3-cf5e-48ba-9c87-8bc9b438cdd3 service glance] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:13:59.565394 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0754ccd3-cf5e-48ba-9c87-8bc9b438cdd3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:13:59.629242 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0754ccd3-cf5e-48ba-9c87-8bc9b438cdd3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:13:59.630845 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0754ccd3-cf5e-48ba-9c87-8bc9b438cdd3 service glance] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:13:59.642472 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1165/2333] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:13:59 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:14:00.011176 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81ddd5e-638c-48b2-ae34-33e03c6996ca None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:14:00.011463 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81ddd5e-638c-48b2-ae34-33e03c6996ca None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:14:00.011738 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81ddd5e-638c-48b2-ae34-33e03c6996ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:14:00.033166 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-f81ddd5e-638c-48b2-ae34-33e03c6996ca None None] MFA Rules not processed for user `d1b26ba592724dd389584bb800d877a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:14:00.034732 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-f81ddd5e-638c-48b2-ae34-33e03c6996ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:14:00.113534 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1169/2334] 10.222.0.18 () {58 vars in 1009 bytes} [Sat Aug 6 22:14:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:14:00.128619 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0754ccd3-cf5e-48ba-9c87-8bc9b438cdd3 service glance] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:14:00.130141 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-0754ccd3-cf5e-48ba-9c87-8bc9b438cdd3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:14:00.162756 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:14:00.205057 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0754ccd3-cf5e-48ba-9c87-8bc9b438cdd3 service glance] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:14:00.210401 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68103dd0-3af3-46f3-aa23-338b63192bb6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7fc9da7505744608dfd3bbd46a0a9d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:14:00.211016 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68103dd0-3af3-46f3-aa23-338b63192bb6 service swift] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:14:00.211316 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68103dd0-3af3-46f3-aa23-338b63192bb6 service swift] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:14:00.211799 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68103dd0-3af3-46f3-aa23-338b63192bb6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:14:00.220965 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68103dd0-3af3-46f3-aa23-338b63192bb6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:14:00.222564 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68103dd0-3af3-46f3-aa23-338b63192bb6 service swift] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:14:00.233568 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1166/2335] 10.222.0.18 () {60 vars in 1384 bytes} [Sat Aug 6 22:14:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 22:14:08.841747 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81ddd5e-638c-48b2-ae34-33e03c6996ca None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:14:08.907581 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81ddd5e-638c-48b2-ae34-33e03c6996ca None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:14:08.913649 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c64d57b-9317-4958-a54b-869fe8e23aea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:14:08.914376 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c64d57b-9317-4958-a54b-869fe8e23aea service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:14:08.914659 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c64d57b-9317-4958-a54b-869fe8e23aea service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:14:08.914934 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c64d57b-9317-4958-a54b-869fe8e23aea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:14:08.916882 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-1c64d57b-9317-4958-a54b-869fe8e23aea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:14:08.940839 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:14:09.008145 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c64d57b-9317-4958-a54b-869fe8e23aea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:14:09.009791 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c64d57b-9317-4958-a54b-869fe8e23aea service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:14:09.020673 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1170/2336] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:14:08 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:14:10.230386 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=20 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:14:11.427470 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c6c60e-fb5f-4a5b-a3a9-3950ee8f6caf None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:14:11.428101 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c6c60e-fb5f-4a5b-a3a9-3950ee8f6caf None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:14:11.428101 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c6c60e-fb5f-4a5b-a3a9-3950ee8f6caf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:14:11.447290 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-86c6c60e-fb5f-4a5b-a3a9-3950ee8f6caf None None] MFA Rules not processed for user `7a851cc02ef54ebc9101964bb5c0d9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:14:11.448528 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-86c6c60e-fb5f-4a5b-a3a9-3950ee8f6caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:14:11.457326 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1167/2337] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:14:11 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:14:13.620937 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb824c-63b7-49ec-85ee-74f3ab2e2479 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:14:13.621448 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb824c-63b7-49ec-85ee-74f3ab2e2479 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:14:13.621568 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb824c-63b7-49ec-85ee-74f3ab2e2479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:14:13.641020 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-d2cb824c-63b7-49ec-85ee-74f3ab2e2479 None None] MFA Rules not processed for user `56138f43f00e46ea9dda399a0fb0e822`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:14:13.643145 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-d2cb824c-63b7-49ec-85ee-74f3ab2e2479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:14:13.706572 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1171/2338] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:14:13 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:14:21.442009 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=3 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:14:23.407918 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be4a0f3-6c68-4f41-8cfd-bc916250ed8a None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:14:23.408943 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be4a0f3-6c68-4f41-8cfd-bc916250ed8a None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:14:23.409222 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be4a0f3-6c68-4f41-8cfd-bc916250ed8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:14:23.429635 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-8be4a0f3-6c68-4f41-8cfd-bc916250ed8a None None] MFA Rules not processed for user `d1b26ba592724dd389584bb800d877a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:14:23.431400 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-8be4a0f3-6c68-4f41-8cfd-bc916250ed8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:14:23.438829 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1168/2339] 10.222.0.18 () {58 vars in 1009 bytes} [Sat Aug 6 22:14:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:14:23.452075 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cb824c-63b7-49ec-85ee-74f3ab2e2479 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:14:23.464893 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cb824c-63b7-49ec-85ee-74f3ab2e2479 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:14:23.470391 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa3a10a-8310-4f74-be04-ae734159bb77 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7fc9da7505744608dfd3bbd46a0a9d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:14:23.471134 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa3a10a-8310-4f74-be04-ae734159bb77 service swift] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:14:23.471483 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa3a10a-8310-4f74-be04-ae734159bb77 service swift] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:14:23.471823 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa3a10a-8310-4f74-be04-ae734159bb77 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:14:23.482075 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa3a10a-8310-4f74-be04-ae734159bb77 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:14:23.483791 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa3a10a-8310-4f74-be04-ae734159bb77 service swift] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:14:23.509653 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1172/2340] 10.222.0.18 () {60 vars in 1384 bytes} [Sat Aug 6 22:14:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 22:14:33.430596 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=3 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:14:33.489184 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=50 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:14:48.524140 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be4a0f3-6c68-4f41-8cfd-bc916250ed8a None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:14:48.525672 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-8be4a0f3-6c68-4f41-8cfd-bc916250ed8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:14:48.591930 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be4a0f3-6c68-4f41-8cfd-bc916250ed8a None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:14:48.596750 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0321a701-8d85-43ca-8a82-2b685c45d1cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:14:48.597330 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0321a701-8d85-43ca-8a82-2b685c45d1cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:14:48.597602 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0321a701-8d85-43ca-8a82-2b685c45d1cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:14:48.597869 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0321a701-8d85-43ca-8a82-2b685c45d1cd admin admin] PATH_INFO: `/v3/users/11db88f3a6ab48f5a2b078d42628d7c3` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:14:48.599002 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0321a701-8d85-43ca-8a82-2b685c45d1cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=11db88f3a6ab48f5a2b078d42628d7c3)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:14:48.602131 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0321a701-8d85-43ca-8a82-2b685c45d1cd admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:14:48.814234 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-0321a701-8d85-43ca-8a82-2b685c45d1cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11db88f3a6ab48f5a2b078d42628d7c3'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:14:48.827075 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-0321a701-8d85-43ca-8a82-2b685c45d1cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11db88f3a6ab48f5a2b078d42628d7c3'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:14:48.902043 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-0321a701-8d85-43ca-8a82-2b685c45d1cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11db88f3a6ab48f5a2b078d42628d7c3'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:14:48.910026 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-0321a701-8d85-43ca-8a82-2b685c45d1cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11db88f3a6ab48f5a2b078d42628d7c3'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:14:48.910742 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-0321a701-8d85-43ca-8a82-2b685c45d1cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11db88f3a6ab48f5a2b078d42628d7c3'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:14:48.918196 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1169/2341] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:14:48 2022] DELETE /identity/v3/users/11db88f3a6ab48f5a2b078d42628d7c3 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:14:48.922691 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d69e6-35b6-40d1-b79a-f463000528b8 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:14:48.923264 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d69e6-35b6-40d1-b79a-f463000528b8 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:14:48.923500 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d69e6-35b6-40d1-b79a-f463000528b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:14:48.946857 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-de5d69e6-35b6-40d1-b79a-f463000528b8 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:14:48.948196 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-de5d69e6-35b6-40d1-b79a-f463000528b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:14:49.009722 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1173/2342] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:14:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:14:49.019312 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0321a701-8d85-43ca-8a82-2b685c45d1cd admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:14:49.020763 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-0321a701-8d85-43ca-8a82-2b685c45d1cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:14:49.085893 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0321a701-8d85-43ca-8a82-2b685c45d1cd admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:14:49.089453 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abe22e5-471a-4142-bae9-1c557c3ec760 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:14:49.090054 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abe22e5-471a-4142-bae9-1c557c3ec760 service neutron] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:14:49.090329 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abe22e5-471a-4142-bae9-1c557c3ec760 service neutron] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:14:49.090599 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abe22e5-471a-4142-bae9-1c557c3ec760 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:14:49.100921 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abe22e5-471a-4142-bae9-1c557c3ec760 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:14:49.102262 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abe22e5-471a-4142-bae9-1c557c3ec760 service neutron] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:14:49.113439 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1170/2343] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:14:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:14:49.915764 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5d69e6-35b6-40d1-b79a-f463000528b8 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:14:49.917280 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-de5d69e6-35b6-40d1-b79a-f463000528b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:14:49.929700 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5d69e6-35b6-40d1-b79a-f463000528b8 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:14:49.934332 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070a3419-f38e-4135-9c26-80eb82a0cf59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:14:49.934953 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a3419-f38e-4135-9c26-80eb82a0cf59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:14:49.935229 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a3419-f38e-4135-9c26-80eb82a0cf59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:14:49.935500 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a3419-f38e-4135-9c26-80eb82a0cf59 admin admin] PATH_INFO: `/v3/projects/43c7e816ab684850a033371d64b8c0c9` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:14:49.936646 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070a3419-f38e-4135-9c26-80eb82a0cf59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43c7e816ab684850a033371d64b8c0c9)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:14:49.938561 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070a3419-f38e-4135-9c26-80eb82a0cf59 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:14:50.035974 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-070a3419-f38e-4135-9c26-80eb82a0cf59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43c7e816ab684850a033371d64b8c0c9', 'request_id': 'req-070a3419-f38e-4135-9c26-80eb82a0cf59'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:14:50.107497 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-070a3419-f38e-4135-9c26-80eb82a0cf59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43c7e816ab684850a033371d64b8c0c9', 'request_id': 'req-070a3419-f38e-4135-9c26-80eb82a0cf59'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:14:50.113832 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-070a3419-f38e-4135-9c26-80eb82a0cf59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43c7e816ab684850a033371d64b8c0c9', 'request_id': 'req-070a3419-f38e-4135-9c26-80eb82a0cf59'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:14:50.125767 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-070a3419-f38e-4135-9c26-80eb82a0cf59 admin admin] The token cache is being invalidate because project 43c7e816ab684850a033371d64b8c0c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:14:50.125960 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-070a3419-f38e-4135-9c26-80eb82a0cf59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:14:50.126723 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-070a3419-f38e-4135-9c26-80eb82a0cf59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:14:50.128394 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1174/2344] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:14:49 2022] DELETE /identity/v3/projects/43c7e816ab684850a033371d64b8c0c9 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:14:50.133925 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e081a8e-2d85-44cd-9e9a-f68791d08489 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:14:50.134450 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e081a8e-2d85-44cd-9e9a-f68791d08489 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:14:50.134586 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e081a8e-2d85-44cd-9e9a-f68791d08489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:14:50.154701 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-1e081a8e-2d85-44cd-9e9a-f68791d08489 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:14:50.155957 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-1e081a8e-2d85-44cd-9e9a-f68791d08489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:14:50.173522 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:14:50.217107 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1171/2345] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:14:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:14:50.225062 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6baaaf-d5bd-4bc8-adfc-6f20efe4db69 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:14:50.225347 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6baaaf-d5bd-4bc8-adfc-6f20efe4db69 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:14:50.225623 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6baaaf-d5bd-4bc8-adfc-6f20efe4db69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:14:50.245086 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-cc6baaaf-d5bd-4bc8-adfc-6f20efe4db69 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:14:50.246302 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-cc6baaaf-d5bd-4bc8-adfc-6f20efe4db69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:14:50.254326 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1175/2346] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:14:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:14:50.267293 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e081a8e-2d85-44cd-9e9a-f68791d08489 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:14:50.278158 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e081a8e-2d85-44cd-9e9a-f68791d08489 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:14:50.282751 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464db5c1-2193-4826-98fc-00d7bd7e1bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:14:50.283345 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464db5c1-2193-4826-98fc-00d7bd7e1bf3 admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:14:50.283621 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464db5c1-2193-4826-98fc-00d7bd7e1bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:14:50.283894 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464db5c1-2193-4826-98fc-00d7bd7e1bf3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:14:50.284581 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464db5c1-2193-4826-98fc-00d7bd7e1bf3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 22:14:50.284924 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464db5c1-2193-4826-98fc-00d7bd7e1bf3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:14:50.286501 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464db5c1-2193-4826-98fc-00d7bd7e1bf3 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:14:50.298811 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1172/2347] 10.222.0.18 () {58 vars in 1153 bytes} [Sat Aug 6 22:14:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 22:14:50.303033 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6baaaf-d5bd-4bc8-adfc-6f20efe4db69 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:14:50.312335 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6baaaf-d5bd-4bc8-adfc-6f20efe4db69 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:14:50.316993 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac0ecbe-ec70-4cc6-bc29-44c5494038fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:14:50.317580 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac0ecbe-ec70-4cc6-bc29-44c5494038fc admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:14:50.318340 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac0ecbe-ec70-4cc6-bc29-44c5494038fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:14:50.318610 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac0ecbe-ec70-4cc6-bc29-44c5494038fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:14:50.319236 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac0ecbe-ec70-4cc6-bc29-44c5494038fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:14:50.321144 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac0ecbe-ec70-4cc6-bc29-44c5494038fc admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:14:50.419274 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1176/2348] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:14:50 2022] POST /identity/v3/projects => generated 369 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:14:50.423421 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464db5c1-2193-4826-98fc-00d7bd7e1bf3 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:14:50.434285 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464db5c1-2193-4826-98fc-00d7bd7e1bf3 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:14:50.495755 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f14ca03-e9c5-4e41-adcf-d7e80ca8558f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:14:50.496360 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f14ca03-e9c5-4e41-adcf-d7e80ca8558f admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:14:50.496667 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f14ca03-e9c5-4e41-adcf-d7e80ca8558f admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:14:50.496933 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f14ca03-e9c5-4e41-adcf-d7e80ca8558f admin admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:14:50.497560 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f14ca03-e9c5-4e41-adcf-d7e80ca8558f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:14:50.499528 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f14ca03-e9c5-4e41-adcf-d7e80ca8558f admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:14:50.581864 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1173/2349] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:14:50 2022] POST /identity/v3/users => generated 285 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:14:50.586496 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac0ecbe-ec70-4cc6-bc29-44c5494038fc admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:14:50.596891 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac0ecbe-ec70-4cc6-bc29-44c5494038fc admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:14:50.601911 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19f3d20-37fc-461d-9964-2e94571ff6ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:14:50.602504 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19f3d20-37fc-461d-9964-2e94571ff6ab admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:14:50.602784 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19f3d20-37fc-461d-9964-2e94571ff6ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:14:50.603056 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19f3d20-37fc-461d-9964-2e94571ff6ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:14:50.603692 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19f3d20-37fc-461d-9964-2e94571ff6ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:14:50.605326 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19f3d20-37fc-461d-9964-2e94571ff6ab admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:14:50.615628 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1177/2350] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:14:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:14:50.619790 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f14ca03-e9c5-4e41-adcf-d7e80ca8558f admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:14:50.630143 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f14ca03-e9c5-4e41-adcf-d7e80ca8558f admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:14:50.635568 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba20d392-9bb2-4d10-a11c-b6fe7b374baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:14:50.636175 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba20d392-9bb2-4d10-a11c-b6fe7b374baf admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:14:50.636472 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba20d392-9bb2-4d10-a11c-b6fe7b374baf admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:14:50.636751 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba20d392-9bb2-4d10-a11c-b6fe7b374baf admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:14:50.637392 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba20d392-9bb2-4d10-a11c-b6fe7b374baf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:14:50.639085 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba20d392-9bb2-4d10-a11c-b6fe7b374baf admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:14:50.649844 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1174/2351] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:14:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:14:50.653815 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19f3d20-37fc-461d-9964-2e94571ff6ab admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:14:50.663808 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19f3d20-37fc-461d-9964-2e94571ff6ab admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:14:50.668372 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbf41ec-fe99-48bf-8b3e-e71f36f4af88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:14:50.669065 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbf41ec-fe99-48bf-8b3e-e71f36f4af88 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:14:50.669397 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbf41ec-fe99-48bf-8b3e-e71f36f4af88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:14:50.669623 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbf41ec-fe99-48bf-8b3e-e71f36f4af88 admin admin] PATH_INFO: `/v3/projects/fd4b1cffaa534155871a5e39da588023/users/ef89afa635d348dc9286960e0c7427dd/roles/56e4a96094e9434cbbafb22579c034f5` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:14:50.684600 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbf41ec-fe99-48bf-8b3e-e71f36f4af88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd4b1cffaa534155871a5e39da588023, user_id=ef89afa635d348dc9286960e0c7427dd, role_id=56e4a96094e9434cbbafb22579c034f5)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:14:50.687277 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbf41ec-fe99-48bf-8b3e-e71f36f4af88 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:14:50.759650 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1178/2352] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:14:50 2022] PUT /identity/v3/projects/fd4b1cffaa534155871a5e39da588023/users/ef89afa635d348dc9286960e0c7427dd/roles/56e4a96094e9434cbbafb22579c034f5 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:14:50.825848 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33647b30-cf04-4409-9149-cab518a62f06 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:14:50.826130 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33647b30-cf04-4409-9149-cab518a62f06 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:14:50.826403 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33647b30-cf04-4409-9149-cab518a62f06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:14:50.855582 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-33647b30-cf04-4409-9149-cab518a62f06 None None] MFA Rules not processed for user `ef89afa635d348dc9286960e0c7427dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:14:50.856823 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-33647b30-cf04-4409-9149-cab518a62f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:14:50.933114 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1175/2353] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:14:50 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:14:50.952316 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbf41ec-fe99-48bf-8b3e-e71f36f4af88 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:14:50.954647 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-1fbf41ec-fe99-48bf-8b3e-e71f36f4af88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:14:50.979990 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:14:51.017095 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbf41ec-fe99-48bf-8b3e-e71f36f4af88 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:14:51.021482 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90034e8d-e894-4972-bd86-e3ab2786b5ed service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8d38e73e105476e8de858cfa939ba20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:14:51.023807 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90034e8d-e894-4972-bd86-e3ab2786b5ed service glance] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:14:51.023807 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90034e8d-e894-4972-bd86-e3ab2786b5ed service glance] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:14:51.023807 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90034e8d-e894-4972-bd86-e3ab2786b5ed service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:14:51.031331 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90034e8d-e894-4972-bd86-e3ab2786b5ed service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:14:51.032794 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90034e8d-e894-4972-bd86-e3ab2786b5ed service glance] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:14:51.042355 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1179/2354] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:14:50 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 22:14:58.144081 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d24ee-aae5-4b5c-a322-f192fff93b35 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:14:58.144555 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d24ee-aae5-4b5c-a322-f192fff93b35 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:14:58.144662 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d24ee-aae5-4b5c-a322-f192fff93b35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:14:58.165840 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-012d24ee-aae5-4b5c-a322-f192fff93b35 None None] MFA Rules not processed for user `d1b26ba592724dd389584bb800d877a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:14:58.167073 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-012d24ee-aae5-4b5c-a322-f192fff93b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:14:58.179975 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:14:58.227372 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1176/2355] 10.222.0.18 () {58 vars in 1009 bytes} [Sat Aug 6 22:14:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:14:58.239989 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90034e8d-e894-4972-bd86-e3ab2786b5ed service glance] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:14:58.241427 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-90034e8d-e894-4972-bd86-e3ab2786b5ed service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:14:58.342726 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90034e8d-e894-4972-bd86-e3ab2786b5ed service glance] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:14:58.352099 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e90e1d3-69be-4ca1-b226-1a226c3a8854 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7fc9da7505744608dfd3bbd46a0a9d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:14:58.352854 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e90e1d3-69be-4ca1-b226-1a226c3a8854 service swift] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:14:58.353157 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e90e1d3-69be-4ca1-b226-1a226c3a8854 service swift] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:14:58.353746 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e90e1d3-69be-4ca1-b226-1a226c3a8854 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:14:58.363163 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e90e1d3-69be-4ca1-b226-1a226c3a8854 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:14:58.364869 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e90e1d3-69be-4ca1-b226-1a226c3a8854 service swift] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:14:58.376645 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1180/2356] 10.222.0.18 () {60 vars in 1384 bytes} [Sat Aug 6 22:14:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 06 22:15:04.520292 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9cc3ef-c386-484f-87df-60b2a534489e None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:15:04.520888 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9cc3ef-c386-484f-87df-60b2a534489e None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:15:04.520888 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9cc3ef-c386-484f-87df-60b2a534489e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:15:04.543319 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-5a9cc3ef-c386-484f-87df-60b2a534489e None None] MFA Rules not processed for user `0d6e35038fed4b44b986902a5ef55ce8`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:15:04.544886 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-5a9cc3ef-c386-484f-87df-60b2a534489e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:15:04.617992 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1177/2357] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:15:04 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:15:08.371005 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=49 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:15:13.578715 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e90e1d3-69be-4ca1-b226-1a226c3a8854 service swift] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:15:13.580252 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-5e90e1d3-69be-4ca1-b226-1a226c3a8854 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:15:13.649136 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e90e1d3-69be-4ca1-b226-1a226c3a8854 service swift] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:15:13.654405 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301f287c-6c35-4350-a58c-133e9f75e3df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:15:13.655108 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f287c-6c35-4350-a58c-133e9f75e3df service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:15:13.655490 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f287c-6c35-4350-a58c-133e9f75e3df service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:15:13.656392 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f287c-6c35-4350-a58c-133e9f75e3df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:15:13.659123 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-301f287c-6c35-4350-a58c-133e9f75e3df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:15:13.755009 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301f287c-6c35-4350-a58c-133e9f75e3df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:15:13.756669 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301f287c-6c35-4350-a58c-133e9f75e3df service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:15:13.769505 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1181/2358] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:15:13 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:15:14.605189 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=42 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:15:15.322275 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a6dd66-a22e-4ca7-bc68-7d307cb7ec77 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:15:15.322559 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a6dd66-a22e-4ca7-bc68-7d307cb7ec77 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:15:15.322863 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a6dd66-a22e-4ca7-bc68-7d307cb7ec77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:15:15.344886 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-23a6dd66-a22e-4ca7-bc68-7d307cb7ec77 None None] MFA Rules not processed for user `700f40b4567a4bcdbb2194861f4497d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:15:15.346197 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-23a6dd66-a22e-4ca7-bc68-7d307cb7ec77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:15:15.416896 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1178/2359] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:15:15 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:15:18.721209 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014a59b3-f938-4813-9866-82110bc869eb None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:15:18.721801 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014a59b3-f938-4813-9866-82110bc869eb None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:15:18.721801 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014a59b3-f938-4813-9866-82110bc869eb None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:15:18.723118 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1182/2360] 10.222.0.18 () {54 vars in 921 bytes} [Sat Aug 6 22:15:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:15:18.730523 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2675075-3b17-4647-a407-4fc940b4fbbb None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:15:18.736932 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2675075-3b17-4647-a407-4fc940b4fbbb None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:15:18.736932 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2675075-3b17-4647-a407-4fc940b4fbbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:15:18.758492 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-b2675075-3b17-4647-a407-4fc940b4fbbb None None] MFA Rules not processed for user `2a1319d33e9144859c6010843be312a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:15:18.759755 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-b2675075-3b17-4647-a407-4fc940b4fbbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:15:18.844030 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1179/2361] 10.222.0.18 () {58 vars in 1018 bytes} [Sat Aug 6 22:15:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:15:18.858963 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014a59b3-f938-4813-9866-82110bc869eb None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:15:18.860708 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-014a59b3-f938-4813-9866-82110bc869eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:15:18.873596 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014a59b3-f938-4813-9866-82110bc869eb None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:15:18.877138 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a38935-a384-4262-90f0-77ce6a3bd6d0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8d38e73e105476e8de858cfa939ba20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:15:18.877725 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a38935-a384-4262-90f0-77ce6a3bd6d0 service glance] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:15:18.882296 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a38935-a384-4262-90f0-77ce6a3bd6d0 service glance] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:15:18.882296 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a38935-a384-4262-90f0-77ce6a3bd6d0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:15:18.888706 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a38935-a384-4262-90f0-77ce6a3bd6d0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:15:18.890119 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a38935-a384-4262-90f0-77ce6a3bd6d0 service glance] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:15:18.901380 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1183/2362] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:15:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:15:19.121857 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c3e50f-e675-4ddb-9789-fe53d54e129a None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:15:19.122170 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c3e50f-e675-4ddb-9789-fe53d54e129a None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:15:19.122453 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c3e50f-e675-4ddb-9789-fe53d54e129a None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:15:19.124014 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1180/2363] 10.222.0.18 () {54 vars in 921 bytes} [Sat Aug 6 22:15:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:15:19.128877 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa18617e-d062-45bb-b1d0-4ff10bb08adf None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:15:19.129164 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa18617e-d062-45bb-b1d0-4ff10bb08adf None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:15:19.129465 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa18617e-d062-45bb-b1d0-4ff10bb08adf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:15:19.151657 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-aa18617e-d062-45bb-b1d0-4ff10bb08adf None None] MFA Rules not processed for user `5ceaeadc843c4186b25114b28607fe22`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:15:19.156987 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-aa18617e-d062-45bb-b1d0-4ff10bb08adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:15:19.164922 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1184/2364] 10.222.0.18 () {58 vars in 1018 bytes} [Sat Aug 6 22:15:19 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:15:19.180870 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c3e50f-e675-4ddb-9789-fe53d54e129a None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:15:19.182720 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-56c3e50f-e675-4ddb-9789-fe53d54e129a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:15:19.196683 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c3e50f-e675-4ddb-9789-fe53d54e129a None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:15:19.200385 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7aee20-d0f3-4a50-9eef-17b647296c86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:15:19.201016 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7aee20-d0f3-4a50-9eef-17b647296c86 service neutron] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:15:19.201297 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7aee20-d0f3-4a50-9eef-17b647296c86 service neutron] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:15:19.201568 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7aee20-d0f3-4a50-9eef-17b647296c86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:15:19.208612 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7aee20-d0f3-4a50-9eef-17b647296c86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:15:19.210170 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7aee20-d0f3-4a50-9eef-17b647296c86 service neutron] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:15:19.218036 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1181/2365] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:15:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:15:19.305288 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdc2ae1-af7c-4e49-9c03-c426078e5c8d None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:15:19.305571 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdc2ae1-af7c-4e49-9c03-c426078e5c8d None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:15:19.305858 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdc2ae1-af7c-4e49-9c03-c426078e5c8d None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:15:19.309937 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1185/2366] 10.222.0.18 () {54 vars in 921 bytes} [Sat Aug 6 22:15:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:15:19.312391 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9199342b-3e55-4876-b7d2-02aef746439a None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:15:19.312725 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9199342b-3e55-4876-b7d2-02aef746439a None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:15:19.312986 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9199342b-3e55-4876-b7d2-02aef746439a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:15:19.333613 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-9199342b-3e55-4876-b7d2-02aef746439a None None] MFA Rules not processed for user `2a1319d33e9144859c6010843be312a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:15:19.334858 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-9199342b-3e55-4876-b7d2-02aef746439a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:15:19.344962 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1182/2367] 10.222.0.18 () {58 vars in 1018 bytes} [Sat Aug 6 22:15:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:15:19.357038 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdc2ae1-af7c-4e49-9c03-c426078e5c8d None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:15:19.368722 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdc2ae1-af7c-4e49-9c03-c426078e5c8d None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:15:19.372299 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d507f5b-1833-41b3-bd2a-04a2083e8485 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:15:19.372926 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d507f5b-1833-41b3-bd2a-04a2083e8485 service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:15:19.373203 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d507f5b-1833-41b3-bd2a-04a2083e8485 service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:15:19.373492 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d507f5b-1833-41b3-bd2a-04a2083e8485 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:15:19.384107 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d507f5b-1833-41b3-bd2a-04a2083e8485 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:15:19.385585 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d507f5b-1833-41b3-bd2a-04a2083e8485 service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:15:19.396317 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1186/2368] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:15:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:15:20.035114 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9199342b-3e55-4876-b7d2-02aef746439a None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:15:20.045220 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9199342b-3e55-4876-b7d2-02aef746439a None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:15:20.049019 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abd1aa7-6f01-4166-ae28-8c734c411213 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:15:20.049619 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abd1aa7-6f01-4166-ae28-8c734c411213 service neutron] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:15:20.049891 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abd1aa7-6f01-4166-ae28-8c734c411213 service neutron] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:15:20.050159 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abd1aa7-6f01-4166-ae28-8c734c411213 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:15:20.051915 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-9abd1aa7-6f01-4166-ae28-8c734c411213 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:15:20.065685 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abd1aa7-6f01-4166-ae28-8c734c411213 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:15:20.067138 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abd1aa7-6f01-4166-ae28-8c734c411213 service neutron] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:15:20.079970 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1183/2369] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:15:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:15:20.526905 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d507f5b-1833-41b3-bd2a-04a2083e8485 service neutron] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:15:20.537247 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d507f5b-1833-41b3-bd2a-04a2083e8485 service neutron] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:15:20.541277 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1518fb-f513-4a01-a95f-5c4190dc4337 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:15:20.542186 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1518fb-f513-4a01-a95f-5c4190dc4337 service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:15:20.542709 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1518fb-f513-4a01-a95f-5c4190dc4337 service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:15:20.543139 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1518fb-f513-4a01-a95f-5c4190dc4337 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:15:20.545076 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-8a1518fb-f513-4a01-a95f-5c4190dc4337 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:15:20.560500 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1518fb-f513-4a01-a95f-5c4190dc4337 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:15:20.562135 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1518fb-f513-4a01-a95f-5c4190dc4337 service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:15:20.574998 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1187/2370] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:15:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:15:29.978813 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abd1aa7-6f01-4166-ae28-8c734c411213 service neutron] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:15:29.980187 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-9abd1aa7-6f01-4166-ae28-8c734c411213 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:15:29.994027 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abd1aa7-6f01-4166-ae28-8c734c411213 service neutron] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:15:29.999739 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c04362e-8076-4ca5-940a-a107676dde88 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a1319d33e9144859c6010843be312a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:15:30.000321 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c04362e-8076-4ca5-940a-a107676dde88 service nova] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:15:30.000630 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c04362e-8076-4ca5-940a-a107676dde88 service nova] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:15:30.000897 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c04362e-8076-4ca5-940a-a107676dde88 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:15:30.002580 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-5c04362e-8076-4ca5-940a-a107676dde88 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:15:30.017358 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c04362e-8076-4ca5-940a-a107676dde88 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:15:30.017358 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c04362e-8076-4ca5-940a-a107676dde88 service nova] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:15:30.024884 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1184/2371] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:15:29 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:15:30.569272 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=71 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:15:32.810698 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1518fb-f513-4a01-a95f-5c4190dc4337 service neutron] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:15:32.811735 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-8a1518fb-f513-4a01-a95f-5c4190dc4337 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:15:32.830854 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1518fb-f513-4a01-a95f-5c4190dc4337 service neutron] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:15:32.844339 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299032ad-940f-47ab-9c93-a86dd9840192 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a1319d33e9144859c6010843be312a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:15:32.845008 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299032ad-940f-47ab-9c93-a86dd9840192 service nova] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:15:32.845294 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299032ad-940f-47ab-9c93-a86dd9840192 service nova] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:15:32.845982 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299032ad-940f-47ab-9c93-a86dd9840192 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:15:32.852067 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-299032ad-940f-47ab-9c93-a86dd9840192 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:15:32.975024 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299032ad-940f-47ab-9c93-a86dd9840192 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:15:32.977372 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299032ad-940f-47ab-9c93-a86dd9840192 service nova] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:15:32.990201 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1188/2372] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:15:32 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:15:40.019794 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=77 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:15:42.984177 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=28 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:15:43.091279 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76507367-16fa-4a67-bd27-5274182afa98 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:15:43.091689 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76507367-16fa-4a67-bd27-5274182afa98 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:15:43.091873 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76507367-16fa-4a67-bd27-5274182afa98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:15:43.111534 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-76507367-16fa-4a67-bd27-5274182afa98 None None] MFA Rules not processed for user `d1b26ba592724dd389584bb800d877a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:15:43.116781 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-76507367-16fa-4a67-bd27-5274182afa98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:15:43.120645 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1185/2373] 10.222.0.18 () {58 vars in 1009 bytes} [Sat Aug 6 22:15:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:15:43.145758 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299032ad-940f-47ab-9c93-a86dd9840192 service nova] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:15:43.154233 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299032ad-940f-47ab-9c93-a86dd9840192 service nova] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:15:43.159206 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba59aa2-a158-4a26-a4b7-2ad764255a03 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7fc9da7505744608dfd3bbd46a0a9d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:15:43.159734 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba59aa2-a158-4a26-a4b7-2ad764255a03 service swift] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:15:43.160009 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba59aa2-a158-4a26-a4b7-2ad764255a03 service swift] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:15:43.160277 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba59aa2-a158-4a26-a4b7-2ad764255a03 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:15:43.168456 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba59aa2-a158-4a26-a4b7-2ad764255a03 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:15:43.170029 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba59aa2-a158-4a26-a4b7-2ad764255a03 service swift] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:15:43.179160 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1189/2374] 10.222.0.18 () {60 vars in 1384 bytes} [Sat Aug 6 22:15:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 22:15:48.746625 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76507367-16fa-4a67-bd27-5274182afa98 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:15:48.812757 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76507367-16fa-4a67-bd27-5274182afa98 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:15:48.817383 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cdec73-949b-48cb-a0cc-512fd1321512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:15:48.818671 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cdec73-949b-48cb-a0cc-512fd1321512 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:15:48.819067 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cdec73-949b-48cb-a0cc-512fd1321512 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:15:48.819752 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cdec73-949b-48cb-a0cc-512fd1321512 admin admin] PATH_INFO: `/v3/users/ef89afa635d348dc9286960e0c7427dd` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:15:48.821074 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cdec73-949b-48cb-a0cc-512fd1321512 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef89afa635d348dc9286960e0c7427dd)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:15:48.823105 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cdec73-949b-48cb-a0cc-512fd1321512 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:15:49.034934 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-c5cdec73-949b-48cb-a0cc-512fd1321512 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef89afa635d348dc9286960e0c7427dd'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:15:49.047600 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-c5cdec73-949b-48cb-a0cc-512fd1321512 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef89afa635d348dc9286960e0c7427dd'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:15:49.191574 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-c5cdec73-949b-48cb-a0cc-512fd1321512 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef89afa635d348dc9286960e0c7427dd'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:15:49.199142 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-c5cdec73-949b-48cb-a0cc-512fd1321512 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef89afa635d348dc9286960e0c7427dd'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:15:49.199775 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-c5cdec73-949b-48cb-a0cc-512fd1321512 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef89afa635d348dc9286960e0c7427dd'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:15:49.205669 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1186/2375] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:15:48 2022] DELETE /identity/v3/users/ef89afa635d348dc9286960e0c7427dd => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:15:49.210585 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69240cd5-3f6c-4e04-b7d6-e96326f18f17 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:15:49.211050 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69240cd5-3f6c-4e04-b7d6-e96326f18f17 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:15:49.211143 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69240cd5-3f6c-4e04-b7d6-e96326f18f17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:15:49.235186 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-69240cd5-3f6c-4e04-b7d6-e96326f18f17 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:15:49.236590 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-69240cd5-3f6c-4e04-b7d6-e96326f18f17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:15:49.300387 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1190/2376] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:15:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:15:49.310200 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cdec73-949b-48cb-a0cc-512fd1321512 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:15:49.311642 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-c5cdec73-949b-48cb-a0cc-512fd1321512 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:15:49.384845 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cdec73-949b-48cb-a0cc-512fd1321512 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:15:49.388369 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db18885e-fea9-461b-9044-ecdd9e32de04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:15:49.388995 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db18885e-fea9-461b-9044-ecdd9e32de04 service neutron] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:15:49.389273 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db18885e-fea9-461b-9044-ecdd9e32de04 service neutron] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:15:49.389546 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db18885e-fea9-461b-9044-ecdd9e32de04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:15:49.398391 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db18885e-fea9-461b-9044-ecdd9e32de04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:15:49.399805 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db18885e-fea9-461b-9044-ecdd9e32de04 service neutron] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:15:49.409060 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1187/2377] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:15:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:15:49.903934 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69240cd5-3f6c-4e04-b7d6-e96326f18f17 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:15:49.906366 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-69240cd5-3f6c-4e04-b7d6-e96326f18f17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:15:49.920333 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69240cd5-3f6c-4e04-b7d6-e96326f18f17 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:15:49.925479 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adbe558-fdac-4f49-a1b5-1cf1960be105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:15:49.926180 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adbe558-fdac-4f49-a1b5-1cf1960be105 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:15:49.926459 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adbe558-fdac-4f49-a1b5-1cf1960be105 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:15:49.926728 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adbe558-fdac-4f49-a1b5-1cf1960be105 admin admin] PATH_INFO: `/v3/projects/fd4b1cffaa534155871a5e39da588023` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:15:49.928517 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adbe558-fdac-4f49-a1b5-1cf1960be105 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd4b1cffaa534155871a5e39da588023)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:15:49.930674 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adbe558-fdac-4f49-a1b5-1cf1960be105 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:15:50.075660 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-5adbe558-fdac-4f49-a1b5-1cf1960be105 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd4b1cffaa534155871a5e39da588023', 'request_id': 'req-5adbe558-fdac-4f49-a1b5-1cf1960be105'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:15:50.146140 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-5adbe558-fdac-4f49-a1b5-1cf1960be105 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd4b1cffaa534155871a5e39da588023', 'request_id': 'req-5adbe558-fdac-4f49-a1b5-1cf1960be105'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:15:50.153427 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-5adbe558-fdac-4f49-a1b5-1cf1960be105 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd4b1cffaa534155871a5e39da588023', 'request_id': 'req-5adbe558-fdac-4f49-a1b5-1cf1960be105'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:15:50.166476 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-5adbe558-fdac-4f49-a1b5-1cf1960be105 admin admin] The token cache is being invalidate because project fd4b1cffaa534155871a5e39da588023 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:15:50.166764 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-5adbe558-fdac-4f49-a1b5-1cf1960be105 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:15:50.167431 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-5adbe558-fdac-4f49-a1b5-1cf1960be105 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:15:50.169043 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1191/2378] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:15:49 2022] DELETE /identity/v3/projects/fd4b1cffaa534155871a5e39da588023 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:15:59.405023 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=82,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:16:00.162948 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=60,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:16:16.591007 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db18885e-fea9-461b-9044-ecdd9e32de04 service neutron] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:16:16.600889 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db18885e-fea9-461b-9044-ecdd9e32de04 service neutron] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:16:16.651973 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f95f186-25d1-4e3c-bf74-2f5e89e82452 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:16:16.652619 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f95f186-25d1-4e3c-bf74-2f5e89e82452 service neutron] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:16:16.652896 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f95f186-25d1-4e3c-bf74-2f5e89e82452 service neutron] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:16:16.653165 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f95f186-25d1-4e3c-bf74-2f5e89e82452 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:16:16.655064 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-4f95f186-25d1-4e3c-bf74-2f5e89e82452 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:16:16.664873 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f95f186-25d1-4e3c-bf74-2f5e89e82452 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:16:16.666322 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f95f186-25d1-4e3c-bf74-2f5e89e82452 service neutron] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:16:16.674890 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1188/2379] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:16:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:16:21.872357 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adbe558-fdac-4f49-a1b5-1cf1960be105 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:16:21.874438 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-5adbe558-fdac-4f49-a1b5-1cf1960be105 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:16:21.967185 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adbe558-fdac-4f49-a1b5-1cf1960be105 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:16:21.973541 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9318d5b-ff17-4865-9c4b-e74986e1d56c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a1319d33e9144859c6010843be312a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:16:21.974466 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9318d5b-ff17-4865-9c4b-e74986e1d56c service nova] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:16:21.974748 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9318d5b-ff17-4865-9c4b-e74986e1d56c service nova] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:16:21.975022 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9318d5b-ff17-4865-9c4b-e74986e1d56c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:16:21.977351 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-a9318d5b-ff17-4865-9c4b-e74986e1d56c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:16:21.991415 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9318d5b-ff17-4865-9c4b-e74986e1d56c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:16:21.993085 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9318d5b-ff17-4865-9c4b-e74986e1d56c service nova] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:16:22.005508 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1192/2380] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:16:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:16:25.260450 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f95f186-25d1-4e3c-bf74-2f5e89e82452 service neutron] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:16:25.261931 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-4f95f186-25d1-4e3c-bf74-2f5e89e82452 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:16:25.278235 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f95f186-25d1-4e3c-bf74-2f5e89e82452 service neutron] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:16:25.283737 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a8d58c-a148-4441-b35d-60d519395644 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a1319d33e9144859c6010843be312a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:16:25.284329 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a8d58c-a148-4441-b35d-60d519395644 service nova] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:16:25.284640 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a8d58c-a148-4441-b35d-60d519395644 service nova] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:16:25.284978 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a8d58c-a148-4441-b35d-60d519395644 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:16:25.287233 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-80a8d58c-a148-4441-b35d-60d519395644 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:16:25.299073 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a8d58c-a148-4441-b35d-60d519395644 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:16:25.300796 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a8d58c-a148-4441-b35d-60d519395644 service nova] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:16:25.311785 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1189/2381] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:16:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:16:28.666579 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710933d1-e673-43c6-b2d8-170958c2afb3 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:16:28.666579 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710933d1-e673-43c6-b2d8-170958c2afb3 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:16:28.666579 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710933d1-e673-43c6-b2d8-170958c2afb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:16:28.685567 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-710933d1-e673-43c6-b2d8-170958c2afb3 None None] MFA Rules not processed for user `d1b26ba592724dd389584bb800d877a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:16:28.686844 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-710933d1-e673-43c6-b2d8-170958c2afb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:16:28.742232 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1193/2382] 10.222.0.18 () {58 vars in 967 bytes} [Sat Aug 6 22:16:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:16:28.755818 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a8d58c-a148-4441-b35d-60d519395644 service nova] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:16:28.757236 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-80a8d58c-a148-4441-b35d-60d519395644 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:16:28.826896 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a8d58c-a148-4441-b35d-60d519395644 service nova] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:16:28.832071 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a726fee-7717-4acf-a9fc-5731866743d2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7fc9da7505744608dfd3bbd46a0a9d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:16:28.832783 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a726fee-7717-4acf-a9fc-5731866743d2 service swift] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:16:28.833065 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a726fee-7717-4acf-a9fc-5731866743d2 service swift] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:16:28.833339 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a726fee-7717-4acf-a9fc-5731866743d2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:16:28.841248 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a726fee-7717-4acf-a9fc-5731866743d2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:16:28.842822 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a726fee-7717-4acf-a9fc-5731866743d2 service swift] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:16:28.851098 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1190/2383] 10.222.0.18 () {60 vars in 1384 bytes} [Sat Aug 6 22:16:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 22:16:38.734945 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=59 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:16:38.847574 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=64 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:16:46.127940 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710933d1-e673-43c6-b2d8-170958c2afb3 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:16:46.127940 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-710933d1-e673-43c6-b2d8-170958c2afb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:16:46.183181 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710933d1-e673-43c6-b2d8-170958c2afb3 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:16:46.187112 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57975382-0d01-4442-ac25-d439597041b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa71c71de263478da234f3c5716a6e5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:16:46.187826 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57975382-0d01-4442-ac25-d439597041b1 service cinder] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:16:46.187985 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57975382-0d01-4442-ac25-d439597041b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:16:46.188279 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57975382-0d01-4442-ac25-d439597041b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:16:46.190014 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-57975382-0d01-4442-ac25-d439597041b1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:16:46.250799 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57975382-0d01-4442-ac25-d439597041b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:16:46.252242 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57975382-0d01-4442-ac25-d439597041b1 service cinder] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:16:46.261368 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1194/2384] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:16:46 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:16:56.257016 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=48 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:16:56.342866 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a726fee-7717-4acf-a9fc-5731866743d2 service swift] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:16:56.344283 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-4a726fee-7717-4acf-a9fc-5731866743d2 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:16:56.357847 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a726fee-7717-4acf-a9fc-5731866743d2 service swift] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:16:56.365429 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9024ed4d-de4b-4e1b-acb0-87f5a48e4064 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:16:56.366013 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9024ed4d-de4b-4e1b-acb0-87f5a48e4064 service neutron] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:16:56.366297 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9024ed4d-de4b-4e1b-acb0-87f5a48e4064 service neutron] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:16:56.366576 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9024ed4d-de4b-4e1b-acb0-87f5a48e4064 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:16:56.368487 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-9024ed4d-de4b-4e1b-acb0-87f5a48e4064 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:16:56.438177 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9024ed4d-de4b-4e1b-acb0-87f5a48e4064 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:16:56.439686 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9024ed4d-de4b-4e1b-acb0-87f5a48e4064 service neutron] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:16:56.450026 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1191/2385] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:16:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:16:57.059377 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57975382-0d01-4442-ac25-d439597041b1 service cinder] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:16:57.060880 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-57975382-0d01-4442-ac25-d439597041b1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:16:57.076612 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57975382-0d01-4442-ac25-d439597041b1 service cinder] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:16:57.081604 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6d19f9-63bb-415c-8fb0-46333a3476f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:16:57.082240 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6d19f9-63bb-415c-8fb0-46333a3476f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:16:57.082526 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6d19f9-63bb-415c-8fb0-46333a3476f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:16:57.082804 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6d19f9-63bb-415c-8fb0-46333a3476f0 admin admin] PATH_INFO: `/v3/users/56138f43f00e46ea9dda399a0fb0e822` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:16:57.084005 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6d19f9-63bb-415c-8fb0-46333a3476f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56138f43f00e46ea9dda399a0fb0e822)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:16:57.086053 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6d19f9-63bb-415c-8fb0-46333a3476f0 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:16:57.288980 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-2d6d19f9-63bb-415c-8fb0-46333a3476f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56138f43f00e46ea9dda399a0fb0e822'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:16:57.300810 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-2d6d19f9-63bb-415c-8fb0-46333a3476f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56138f43f00e46ea9dda399a0fb0e822'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:16:57.308088 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-2d6d19f9-63bb-415c-8fb0-46333a3476f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56138f43f00e46ea9dda399a0fb0e822'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:16:57.311298 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-2d6d19f9-63bb-415c-8fb0-46333a3476f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56138f43f00e46ea9dda399a0fb0e822'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:16:57.311298 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-2d6d19f9-63bb-415c-8fb0-46333a3476f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56138f43f00e46ea9dda399a0fb0e822'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:16:57.383922 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1195/2386] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:16:57 2022] DELETE /identity/v3/users/56138f43f00e46ea9dda399a0fb0e822 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:16:57.631193 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9024ed4d-de4b-4e1b-acb0-87f5a48e4064 service neutron] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:16:57.643490 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9024ed4d-de4b-4e1b-acb0-87f5a48e4064 service neutron] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:16:57.702295 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a383761b-1e46-463d-beb8-20f40c8bdca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:16:57.702898 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a383761b-1e46-463d-beb8-20f40c8bdca8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:16:57.703172 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a383761b-1e46-463d-beb8-20f40c8bdca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:16:57.703441 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a383761b-1e46-463d-beb8-20f40c8bdca8 admin admin] PATH_INFO: `/v3/projects/dc2fb805386743d1a4bfd3b15bde695c` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:16:57.704575 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a383761b-1e46-463d-beb8-20f40c8bdca8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc2fb805386743d1a4bfd3b15bde695c)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:16:57.706495 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a383761b-1e46-463d-beb8-20f40c8bdca8 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:16:57.801890 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-a383761b-1e46-463d-beb8-20f40c8bdca8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc2fb805386743d1a4bfd3b15bde695c', 'request_id': 'req-a383761b-1e46-463d-beb8-20f40c8bdca8'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:16:57.863390 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-a383761b-1e46-463d-beb8-20f40c8bdca8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc2fb805386743d1a4bfd3b15bde695c', 'request_id': 'req-a383761b-1e46-463d-beb8-20f40c8bdca8'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:16:57.869209 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-a383761b-1e46-463d-beb8-20f40c8bdca8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc2fb805386743d1a4bfd3b15bde695c', 'request_id': 'req-a383761b-1e46-463d-beb8-20f40c8bdca8'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:16:57.880229 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-a383761b-1e46-463d-beb8-20f40c8bdca8 admin admin] The token cache is being invalidate because project dc2fb805386743d1a4bfd3b15bde695c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:16:57.880484 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-a383761b-1e46-463d-beb8-20f40c8bdca8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:16:57.881111 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-a383761b-1e46-463d-beb8-20f40c8bdca8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:16:57.882658 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1192/2387] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:16:57 2022] DELETE /identity/v3/projects/dc2fb805386743d1a4bfd3b15bde695c => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:16:57.887613 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba4e864-bfb9-4d06-a8a5-62987c8538bf None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:16:57.887899 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba4e864-bfb9-4d06-a8a5-62987c8538bf None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:16:57.888175 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba4e864-bfb9-4d06-a8a5-62987c8538bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:16:57.909741 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-9ba4e864-bfb9-4d06-a8a5-62987c8538bf None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:16:57.910934 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-9ba4e864-bfb9-4d06-a8a5-62987c8538bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:16:57.969620 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1196/2388] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:16:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:16:57.979445 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8537d1-6cea-4479-9fe5-85b280965015 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:16:57.979725 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8537d1-6cea-4479-9fe5-85b280965015 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:16:57.980013 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8537d1-6cea-4479-9fe5-85b280965015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:16:57.999250 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-bf8537d1-6cea-4479-9fe5-85b280965015 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:16:58.000465 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-bf8537d1-6cea-4479-9fe5-85b280965015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:16:58.008204 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1193/2389] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:16:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:16:58.012329 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba4e864-bfb9-4d06-a8a5-62987c8538bf None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:16:58.021824 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba4e864-bfb9-4d06-a8a5-62987c8538bf None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:16:58.026859 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01395f83-016e-45c3-a0ff-1b727d76c5c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:16:58.027440 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01395f83-016e-45c3-a0ff-1b727d76c5c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:16:58.027715 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01395f83-016e-45c3-a0ff-1b727d76c5c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:16:58.027984 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01395f83-016e-45c3-a0ff-1b727d76c5c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:16:58.028658 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01395f83-016e-45c3-a0ff-1b727d76c5c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 22:16:58.028958 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01395f83-016e-45c3-a0ff-1b727d76c5c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:16:58.030568 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01395f83-016e-45c3-a0ff-1b727d76c5c9 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:16:58.042400 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1197/2390] 10.222.0.18 () {58 vars in 1153 bytes} [Sat Aug 6 22:16:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 22:16:58.046621 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8537d1-6cea-4479-9fe5-85b280965015 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:16:58.055907 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8537d1-6cea-4479-9fe5-85b280965015 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:16:58.060489 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28962e73-08b3-4776-b26d-57c339adc22b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:16:58.061067 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28962e73-08b3-4776-b26d-57c339adc22b admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:16:58.061361 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28962e73-08b3-4776-b26d-57c339adc22b admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:16:58.061614 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28962e73-08b3-4776-b26d-57c339adc22b admin admin] PATH_INFO: `/v3/projects` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:16:58.062262 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28962e73-08b3-4776-b26d-57c339adc22b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:16:58.064135 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28962e73-08b3-4776-b26d-57c339adc22b admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:16:58.080666 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:16:58.157429 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1194/2391] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:16:58 2022] POST /identity/v3/projects => generated 399 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:16:58.162075 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01395f83-016e-45c3-a0ff-1b727d76c5c9 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:16:58.172059 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01395f83-016e-45c3-a0ff-1b727d76c5c9 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:16:58.227753 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3ac391-256d-4e77-a6c7-9e393c8f6fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:16:58.228659 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ac391-256d-4e77-a6c7-9e393c8f6fb1 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:16:58.229073 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ac391-256d-4e77-a6c7-9e393c8f6fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:16:58.229460 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ac391-256d-4e77-a6c7-9e393c8f6fb1 admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:16:58.230093 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3ac391-256d-4e77-a6c7-9e393c8f6fb1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:16:58.231990 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3ac391-256d-4e77-a6c7-9e393c8f6fb1 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:16:58.239811 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:16:58.321286 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1198/2392] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:16:58 2022] POST /identity/v3/users => generated 300 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:16:58.333788 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28962e73-08b3-4776-b26d-57c339adc22b admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:16:58.335787 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28962e73-08b3-4776-b26d-57c339adc22b admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:16:58.340859 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4316f8-968d-4dd6-b445-b6670192259f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:16:58.341444 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4316f8-968d-4dd6-b445-b6670192259f admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:16:58.341719 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4316f8-968d-4dd6-b445-b6670192259f admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:16:58.341986 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4316f8-968d-4dd6-b445-b6670192259f admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:16:58.342619 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4316f8-968d-4dd6-b445-b6670192259f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:16:58.344198 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4316f8-968d-4dd6-b445-b6670192259f admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:16:58.355819 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1195/2393] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:16:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:16:58.359969 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3ac391-256d-4e77-a6c7-9e393c8f6fb1 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:16:58.370226 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3ac391-256d-4e77-a6c7-9e393c8f6fb1 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:16:58.375098 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac3bb31-4ccd-4b85-bba4-e64aa475a9f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:16:58.375689 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac3bb31-4ccd-4b85-bba4-e64aa475a9f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:16:58.375969 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac3bb31-4ccd-4b85-bba4-e64aa475a9f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:16:58.376241 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac3bb31-4ccd-4b85-bba4-e64aa475a9f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:16:58.377032 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac3bb31-4ccd-4b85-bba4-e64aa475a9f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:16:58.379352 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac3bb31-4ccd-4b85-bba4-e64aa475a9f2 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:16:58.390488 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1199/2394] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:16:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:16:58.394471 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4316f8-968d-4dd6-b445-b6670192259f admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:16:58.404985 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4316f8-968d-4dd6-b445-b6670192259f admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:16:58.409614 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f04a92-cb40-4f7e-9e69-4772ea9a527b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:16:58.410228 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f04a92-cb40-4f7e-9e69-4772ea9a527b admin admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:16:58.410502 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f04a92-cb40-4f7e-9e69-4772ea9a527b admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:16:58.410773 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f04a92-cb40-4f7e-9e69-4772ea9a527b admin admin] PATH_INFO: `/v3/projects/3c9fb0982f964775bc0a7cd656a2981d/users/b043d0a54eec4120bd9fe0833cb0217d/roles/56e4a96094e9434cbbafb22579c034f5` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:16:58.424978 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f04a92-cb40-4f7e-9e69-4772ea9a527b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c9fb0982f964775bc0a7cd656a2981d, user_id=b043d0a54eec4120bd9fe0833cb0217d, role_id=56e4a96094e9434cbbafb22579c034f5)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:16:58.427604 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f04a92-cb40-4f7e-9e69-4772ea9a527b admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:16:58.515950 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1196/2395] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:16:58 2022] PUT /identity/v3/projects/3c9fb0982f964775bc0a7cd656a2981d/users/b043d0a54eec4120bd9fe0833cb0217d/roles/56e4a96094e9434cbbafb22579c034f5 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:16:58.526465 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c7ea5a-d8d4-4a60-9b58-f559b79686ed None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:16:58.526745 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c7ea5a-d8d4-4a60-9b58-f559b79686ed None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:16:58.527017 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c7ea5a-d8d4-4a60-9b58-f559b79686ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:16:58.554447 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-39c7ea5a-d8d4-4a60-9b58-f559b79686ed None None] MFA Rules not processed for user `b043d0a54eec4120bd9fe0833cb0217d`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:16:58.556139 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-39c7ea5a-d8d4-4a60-9b58-f559b79686ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:16:58.619650 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1200/2396] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:16:58 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:16:58.630919 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f04a92-cb40-4f7e-9e69-4772ea9a527b admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:16:58.632621 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-b1f04a92-cb40-4f7e-9e69-4772ea9a527b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:16:58.690082 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f04a92-cb40-4f7e-9e69-4772ea9a527b admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:16:58.693768 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627e2651-df14-43e4-bca0-55b9de5da179 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa71c71de263478da234f3c5716a6e5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:16:58.694369 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627e2651-df14-43e4-bca0-55b9de5da179 service cinder] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:16:58.694649 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627e2651-df14-43e4-bca0-55b9de5da179 service cinder] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:16:58.694920 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627e2651-df14-43e4-bca0-55b9de5da179 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:16:58.704066 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627e2651-df14-43e4-bca0-55b9de5da179 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:16:58.705524 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627e2651-df14-43e4-bca0-55b9de5da179 service cinder] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:16:58.714818 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1197/2397] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:16:58 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:17:08.615823 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:INSERT=3,keystone:SELECT=39 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:17:08.710169 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=43,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:17:09.669698 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c7ea5a-d8d4-4a60-9b58-f559b79686ed None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:17:09.732600 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c7ea5a-d8d4-4a60-9b58-f559b79686ed None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:17:09.737081 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7947ff76-22c8-4b72-853e-36995559a0e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:17:09.737700 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7947ff76-22c8-4b72-853e-36995559a0e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:17:09.737979 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7947ff76-22c8-4b72-853e-36995559a0e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:17:09.738251 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7947ff76-22c8-4b72-853e-36995559a0e0 admin admin] PATH_INFO: `/v3/users/b043d0a54eec4120bd9fe0833cb0217d` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:17:09.739351 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7947ff76-22c8-4b72-853e-36995559a0e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b043d0a54eec4120bd9fe0833cb0217d)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:17:09.741297 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7947ff76-22c8-4b72-853e-36995559a0e0 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:17:09.923276 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-7947ff76-22c8-4b72-853e-36995559a0e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b043d0a54eec4120bd9fe0833cb0217d'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:17:09.935661 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-7947ff76-22c8-4b72-853e-36995559a0e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b043d0a54eec4120bd9fe0833cb0217d'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:17:09.942508 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-7947ff76-22c8-4b72-853e-36995559a0e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b043d0a54eec4120bd9fe0833cb0217d'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:17:09.943165 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-7947ff76-22c8-4b72-853e-36995559a0e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b043d0a54eec4120bd9fe0833cb0217d'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:17:09.943773 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-7947ff76-22c8-4b72-853e-36995559a0e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b043d0a54eec4120bd9fe0833cb0217d'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:17:10.011908 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1201/2398] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:17:09 2022] DELETE /identity/v3/users/b043d0a54eec4120bd9fe0833cb0217d => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:17:10.016497 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1992a39c-7584-4fd3-a5b6-dc4862698d9a None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:17:10.016972 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1992a39c-7584-4fd3-a5b6-dc4862698d9a None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:17:10.017074 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1992a39c-7584-4fd3-a5b6-dc4862698d9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:17:10.035642 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-1992a39c-7584-4fd3-a5b6-dc4862698d9a None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:17:10.036960 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-1992a39c-7584-4fd3-a5b6-dc4862698d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:17:10.095123 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1198/2399] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:17:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:17:10.105610 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7947ff76-22c8-4b72-853e-36995559a0e0 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:17:10.107123 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-7947ff76-22c8-4b72-853e-36995559a0e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:17:10.165898 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7947ff76-22c8-4b72-853e-36995559a0e0 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:17:10.170028 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f283096b-0db1-4b3c-99c2-0db07979f703 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:17:10.170652 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f283096b-0db1-4b3c-99c2-0db07979f703 service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:17:10.170927 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f283096b-0db1-4b3c-99c2-0db07979f703 service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:17:10.171198 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f283096b-0db1-4b3c-99c2-0db07979f703 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:17:10.180019 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f283096b-0db1-4b3c-99c2-0db07979f703 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:17:10.181395 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f283096b-0db1-4b3c-99c2-0db07979f703 service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:17:10.191177 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1202/2400] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:17:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:17:10.578991 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1992a39c-7584-4fd3-a5b6-dc4862698d9a None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:17:10.580483 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-1992a39c-7584-4fd3-a5b6-dc4862698d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:17:10.594628 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1992a39c-7584-4fd3-a5b6-dc4862698d9a None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:17:10.599461 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9734a66c-b589-4e43-b5a5-9cdc7fdb4a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:17:10.601810 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9734a66c-b589-4e43-b5a5-9cdc7fdb4a43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:17:10.602092 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9734a66c-b589-4e43-b5a5-9cdc7fdb4a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:17:10.602362 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9734a66c-b589-4e43-b5a5-9cdc7fdb4a43 admin admin] PATH_INFO: `/v3/projects/3c9fb0982f964775bc0a7cd656a2981d` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:17:10.603841 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9734a66c-b589-4e43-b5a5-9cdc7fdb4a43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c9fb0982f964775bc0a7cd656a2981d)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:17:10.606468 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9734a66c-b589-4e43-b5a5-9cdc7fdb4a43 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:17:10.697476 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-9734a66c-b589-4e43-b5a5-9cdc7fdb4a43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c9fb0982f964775bc0a7cd656a2981d', 'request_id': 'req-9734a66c-b589-4e43-b5a5-9cdc7fdb4a43'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:17:10.737530 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-9734a66c-b589-4e43-b5a5-9cdc7fdb4a43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c9fb0982f964775bc0a7cd656a2981d', 'request_id': 'req-9734a66c-b589-4e43-b5a5-9cdc7fdb4a43'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:17:10.743321 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-9734a66c-b589-4e43-b5a5-9cdc7fdb4a43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c9fb0982f964775bc0a7cd656a2981d', 'request_id': 'req-9734a66c-b589-4e43-b5a5-9cdc7fdb4a43'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:17:10.754508 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-9734a66c-b589-4e43-b5a5-9cdc7fdb4a43 admin admin] The token cache is being invalidate because project 3c9fb0982f964775bc0a7cd656a2981d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:17:10.754807 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-9734a66c-b589-4e43-b5a5-9cdc7fdb4a43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:17:10.755439 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-9734a66c-b589-4e43-b5a5-9cdc7fdb4a43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:17:10.757005 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1199/2401] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:17:10 2022] DELETE /identity/v3/projects/3c9fb0982f964775bc0a7cd656a2981d => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:17:20.187186 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:17:20.751331 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:17:35.771054 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f283096b-0db1-4b3c-99c2-0db07979f703 service neutron] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:17:35.772555 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-f283096b-0db1-4b3c-99c2-0db07979f703 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:17:35.845390 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f283096b-0db1-4b3c-99c2-0db07979f703 service neutron] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:17:35.906957 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1418e2c9-e50c-45cf-bae6-0d5c06fa930e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a1319d33e9144859c6010843be312a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:17:35.907564 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1418e2c9-e50c-45cf-bae6-0d5c06fa930e service nova] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:17:35.907845 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1418e2c9-e50c-45cf-bae6-0d5c06fa930e service nova] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:17:35.908138 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1418e2c9-e50c-45cf-bae6-0d5c06fa930e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:17:35.909904 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-1418e2c9-e50c-45cf-bae6-0d5c06fa930e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:17:35.966812 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1418e2c9-e50c-45cf-bae6-0d5c06fa930e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:17:35.968432 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1418e2c9-e50c-45cf-bae6-0d5c06fa930e service nova] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:17:35.977856 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1203/2402] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:17:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:17:45.973822 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=84 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:17:52.103963 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9734a66c-b589-4e43-b5a5-9cdc7fdb4a43 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:17:52.105446 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-9734a66c-b589-4e43-b5a5-9cdc7fdb4a43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:17:52.172521 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9734a66c-b589-4e43-b5a5-9cdc7fdb4a43 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:17:52.177250 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2059675-0fea-4a31-9e78-0bf28797e501 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:17:52.177970 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2059675-0fea-4a31-9e78-0bf28797e501 service neutron] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:17:52.178260 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2059675-0fea-4a31-9e78-0bf28797e501 service neutron] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:17:52.178548 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2059675-0fea-4a31-9e78-0bf28797e501 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:17:52.180318 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-e2059675-0fea-4a31-9e78-0bf28797e501 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:17:52.192439 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2059675-0fea-4a31-9e78-0bf28797e501 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:17:52.193904 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2059675-0fea-4a31-9e78-0bf28797e501 service neutron] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:17:52.204991 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1200/2403] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:17:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:17:52.711448 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c813514b-9da8-42fe-ae94-794d7d82c10f None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:17:52.712015 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c813514b-9da8-42fe-ae94-794d7d82c10f None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:17:52.712015 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c813514b-9da8-42fe-ae94-794d7d82c10f None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:17:52.713563 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1204/2404] 10.222.0.18 () {54 vars in 921 bytes} [Sat Aug 6 22:17:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:17:52.720436 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f978756-36e1-4fd6-904a-f39e7cf8398b None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:17:52.720810 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f978756-36e1-4fd6-904a-f39e7cf8398b None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:17:52.721240 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f978756-36e1-4fd6-904a-f39e7cf8398b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:17:52.741629 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-4f978756-36e1-4fd6-904a-f39e7cf8398b None None] MFA Rules not processed for user `5ceaeadc843c4186b25114b28607fe22`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:17:52.743019 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-4f978756-36e1-4fd6-904a-f39e7cf8398b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:17:52.750671 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1201/2405] 10.222.0.18 () {58 vars in 1018 bytes} [Sat Aug 6 22:17:52 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:17:52.763332 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c813514b-9da8-42fe-ae94-794d7d82c10f None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:17:52.765168 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-c813514b-9da8-42fe-ae94-794d7d82c10f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:17:52.779154 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c813514b-9da8-42fe-ae94-794d7d82c10f None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:17:52.796610 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405ccb17-f7b9-4eff-9e47-3450fb3f961b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:17:52.796610 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ccb17-f7b9-4eff-9e47-3450fb3f961b service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:17:52.796610 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ccb17-f7b9-4eff-9e47-3450fb3f961b service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:17:52.796610 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ccb17-f7b9-4eff-9e47-3450fb3f961b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:17:52.797990 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405ccb17-f7b9-4eff-9e47-3450fb3f961b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:17:52.799468 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405ccb17-f7b9-4eff-9e47-3450fb3f961b service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:17:52.808100 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1205/2406] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:17:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:17:52.892714 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955e2d77-f789-4357-ae96-3df55f2ac88a None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:17:52.892714 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955e2d77-f789-4357-ae96-3df55f2ac88a None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:17:52.892988 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955e2d77-f789-4357-ae96-3df55f2ac88a None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:17:52.894346 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1202/2407] 10.222.0.18 () {54 vars in 921 bytes} [Sat Aug 6 22:17:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:17:52.899475 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849000be-face-4360-a9b3-96e30cd64521 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:17:52.899854 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849000be-face-4360-a9b3-96e30cd64521 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:17:52.900240 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849000be-face-4360-a9b3-96e30cd64521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:17:52.924372 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-849000be-face-4360-a9b3-96e30cd64521 None None] MFA Rules not processed for user `2a1319d33e9144859c6010843be312a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:17:52.924372 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-849000be-face-4360-a9b3-96e30cd64521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:17:52.933238 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1206/2408] 10.222.0.18 () {58 vars in 1018 bytes} [Sat Aug 6 22:17:52 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 22:17:52.943689 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955e2d77-f789-4357-ae96-3df55f2ac88a None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:17:52.954663 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955e2d77-f789-4357-ae96-3df55f2ac88a None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:17:52.958292 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e7be49-01b9-4bd6-a16a-8d08d61f0ae5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:17:52.958863 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e7be49-01b9-4bd6-a16a-8d08d61f0ae5 service neutron] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:17:52.959136 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e7be49-01b9-4bd6-a16a-8d08d61f0ae5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:17:52.959404 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e7be49-01b9-4bd6-a16a-8d08d61f0ae5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:17:52.970235 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e7be49-01b9-4bd6-a16a-8d08d61f0ae5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:17:52.975996 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e7be49-01b9-4bd6-a16a-8d08d61f0ae5 service neutron] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:17:52.985109 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1203/2409] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:17:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:17:53.077150 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849000be-face-4360-a9b3-96e30cd64521 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:17:53.078683 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-849000be-face-4360-a9b3-96e30cd64521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:17:53.154755 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849000be-face-4360-a9b3-96e30cd64521 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:17:53.160056 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb61e86-f1b7-4cbc-9e6f-494ebe09264b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '680d62a67ae2456f8c129e26728c3297', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:17:53.160706 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb61e86-f1b7-4cbc-9e6f-494ebe09264b service placement] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:17:53.160989 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb61e86-f1b7-4cbc-9e6f-494ebe09264b service placement] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:17:53.161265 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb61e86-f1b7-4cbc-9e6f-494ebe09264b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:17:53.163006 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-4cb61e86-f1b7-4cbc-9e6f-494ebe09264b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:17:53.175694 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb61e86-f1b7-4cbc-9e6f-494ebe09264b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:17:53.177322 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb61e86-f1b7-4cbc-9e6f-494ebe09264b service placement] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:17:53.187439 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1207/2410] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:17:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:17:54.567721 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f108157-d095-489d-a31d-6b04d25f2c5f None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:17:54.568238 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f108157-d095-489d-a31d-6b04d25f2c5f None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:17:54.568238 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f108157-d095-489d-a31d-6b04d25f2c5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:17:54.586667 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-3f108157-d095-489d-a31d-6b04d25f2c5f None None] MFA Rules not processed for user `dfc4de8c76d0470a892dc8bad4d29441`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:17:54.590092 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-3f108157-d095-489d-a31d-6b04d25f2c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:17:54.643821 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1204/2411] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:17:54 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:17:55.000536 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb61e86-f1b7-4cbc-9e6f-494ebe09264b service placement] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:17:55.002009 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-4cb61e86-f1b7-4cbc-9e6f-494ebe09264b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:17:55.019311 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb61e86-f1b7-4cbc-9e6f-494ebe09264b service placement] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:17:55.026142 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8537be-9d1e-46e8-831a-3e7fa57f37e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a1319d33e9144859c6010843be312a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:17:55.026857 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8537be-9d1e-46e8-831a-3e7fa57f37e5 service nova] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:17:55.027135 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8537be-9d1e-46e8-831a-3e7fa57f37e5 service nova] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:17:55.027404 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8537be-9d1e-46e8-831a-3e7fa57f37e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:17:55.029232 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-ae8537be-9d1e-46e8-831a-3e7fa57f37e5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:17:55.102146 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8537be-9d1e-46e8-831a-3e7fa57f37e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:17:55.102146 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8537be-9d1e-46e8-831a-3e7fa57f37e5 service nova] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:17:55.110585 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1208/2412] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:17:54 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:17:56.771886 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9032933-d065-4404-b066-28eee8b70be7 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:17:56.772364 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9032933-d065-4404-b066-28eee8b70be7 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:17:56.772498 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9032933-d065-4404-b066-28eee8b70be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:17:56.800691 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-e9032933-d065-4404-b066-28eee8b70be7 None None] MFA Rules not processed for user `0d6e35038fed4b44b986902a5ef55ce8`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:17:56.802454 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-e9032933-d065-4404-b066-28eee8b70be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:17:56.867282 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1205/2413] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:17:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:17:57.235321 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8537be-9d1e-46e8-831a-3e7fa57f37e5 service nova] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:17:57.245024 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8537be-9d1e-46e8-831a-3e7fa57f37e5 service nova] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:17:57.248858 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4695b3b-98a3-4aed-9376-b5fdfccd2e3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:17:57.249460 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4695b3b-98a3-4aed-9376-b5fdfccd2e3e service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:17:57.249745 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4695b3b-98a3-4aed-9376-b5fdfccd2e3e service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:17:57.250016 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4695b3b-98a3-4aed-9376-b5fdfccd2e3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:17:57.251780 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-f4695b3b-98a3-4aed-9376-b5fdfccd2e3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:17:57.262245 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4695b3b-98a3-4aed-9376-b5fdfccd2e3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:17:57.263674 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4695b3b-98a3-4aed-9376-b5fdfccd2e3e service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:17:57.271563 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1209/2414] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:17:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:18:06.860543 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=85 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:18:07.268037 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=75 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:18:16.653222 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9032933-d065-4404-b066-28eee8b70be7 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:18:16.654711 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-e9032933-d065-4404-b066-28eee8b70be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:18:16.671893 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9032933-d065-4404-b066-28eee8b70be7 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:18:16.677314 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b661aa6-896f-4dd8-ba1e-c69a9aba4b67 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a1319d33e9144859c6010843be312a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:18:16.677914 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b661aa6-896f-4dd8-ba1e-c69a9aba4b67 service nova] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:18:16.678192 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b661aa6-896f-4dd8-ba1e-c69a9aba4b67 service nova] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:18:16.678455 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b661aa6-896f-4dd8-ba1e-c69a9aba4b67 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:18:16.680236 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-9b661aa6-896f-4dd8-ba1e-c69a9aba4b67 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:18:16.693364 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b661aa6-896f-4dd8-ba1e-c69a9aba4b67 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:18:16.694929 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b661aa6-896f-4dd8-ba1e-c69a9aba4b67 service nova] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:18:16.706389 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1206/2415] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:18:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:18:26.701754 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=6 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:18:39.881964 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e0dfe0-8b0b-42b7-8b0f-6c1ed36369ff None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:18:39.882505 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e0dfe0-8b0b-42b7-8b0f-6c1ed36369ff None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:18:39.882505 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e0dfe0-8b0b-42b7-8b0f-6c1ed36369ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:18:39.910561 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-66e0dfe0-8b0b-42b7-8b0f-6c1ed36369ff None None] MFA Rules not processed for user `700f40b4567a4bcdbb2194861f4497d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:18:39.911744 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-66e0dfe0-8b0b-42b7-8b0f-6c1ed36369ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:18:39.935439 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1210/2416] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:18:39 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:18:49.933409 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=15 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:18:51.244219 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b661aa6-896f-4dd8-ba1e-c69a9aba4b67 service nova] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:18:51.308094 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b661aa6-896f-4dd8-ba1e-c69a9aba4b67 service nova] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:18:51.314679 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc97cd66-5a3c-49e6-81a5-cd32d1ed50ba service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '680d62a67ae2456f8c129e26728c3297', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:18:51.318065 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc97cd66-5a3c-49e6-81a5-cd32d1ed50ba service placement] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:18:51.318065 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc97cd66-5a3c-49e6-81a5-cd32d1ed50ba service placement] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:18:51.318065 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc97cd66-5a3c-49e6-81a5-cd32d1ed50ba service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:18:51.318065 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-bc97cd66-5a3c-49e6-81a5-cd32d1ed50ba service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:18:51.347178 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc97cd66-5a3c-49e6-81a5-cd32d1ed50ba service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:18:51.349142 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc97cd66-5a3c-49e6-81a5-cd32d1ed50ba service placement] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:18:51.359406 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1207/2417] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:18:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:18:52.509453 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb384902-2fb8-430a-a78e-3d6b76dc0d1c None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:18:52.509940 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb384902-2fb8-430a-a78e-3d6b76dc0d1c None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:18:52.510049 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb384902-2fb8-430a-a78e-3d6b76dc0d1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:18:52.541341 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-eb384902-2fb8-430a-a78e-3d6b76dc0d1c None None] MFA Rules not processed for user `d1b26ba592724dd389584bb800d877a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:18:52.542564 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-eb384902-2fb8-430a-a78e-3d6b76dc0d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:18:52.609898 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1211/2418] 10.222.0.18 () {58 vars in 967 bytes} [Sat Aug 6 22:18:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 22:18:52.784636 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc97cd66-5a3c-49e6-81a5-cd32d1ed50ba service placement] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:18:52.786264 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-bc97cd66-5a3c-49e6-81a5-cd32d1ed50ba service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:18:52.861507 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc97cd66-5a3c-49e6-81a5-cd32d1ed50ba service placement] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:18:52.866487 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503fe8bb-6a2e-4c16-a1fc-fddf6c74ad1c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7fc9da7505744608dfd3bbd46a0a9d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:18:52.867089 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503fe8bb-6a2e-4c16-a1fc-fddf6c74ad1c service swift] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:18:52.867363 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503fe8bb-6a2e-4c16-a1fc-fddf6c74ad1c service swift] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:18:52.867631 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503fe8bb-6a2e-4c16-a1fc-fddf6c74ad1c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:18:52.876463 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503fe8bb-6a2e-4c16-a1fc-fddf6c74ad1c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:18:52.878072 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503fe8bb-6a2e-4c16-a1fc-fddf6c74ad1c service swift] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:18:52.886105 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1208/2419] 10.222.0.18 () {60 vars in 1384 bytes} [Sat Aug 6 22:18:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 22:18:54.145063 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb384902-2fb8-430a-a78e-3d6b76dc0d1c None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:18:54.164373 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb384902-2fb8-430a-a78e-3d6b76dc0d1c None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:18:54.167969 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517e44b6-6ed7-459c-8a8e-4e069d381071 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:18:54.168638 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517e44b6-6ed7-459c-8a8e-4e069d381071 service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:18:54.168913 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517e44b6-6ed7-459c-8a8e-4e069d381071 service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:18:54.169181 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517e44b6-6ed7-459c-8a8e-4e069d381071 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:18:54.171014 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-517e44b6-6ed7-459c-8a8e-4e069d381071 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:18:54.254139 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517e44b6-6ed7-459c-8a8e-4e069d381071 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:18:54.255489 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517e44b6-6ed7-459c-8a8e-4e069d381071 service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:18:54.273757 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1212/2420] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:18:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:18:58.060283 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4960d771-ea0f-47c1-a389-46a40dfd3eb7 None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:18:58.060870 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4960d771-ea0f-47c1-a389-46a40dfd3eb7 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:18:58.060968 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4960d771-ea0f-47c1-a389-46a40dfd3eb7 None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:18:58.062377 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1209/2421] 10.222.0.18 () {54 vars in 921 bytes} [Sat Aug 6 22:18:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:18:58.087759 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6630977-eec7-4646-a572-0f1e3b3c052d None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:18:58.088647 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6630977-eec7-4646-a572-0f1e3b3c052d None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:18:58.088923 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6630977-eec7-4646-a572-0f1e3b3c052d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:18:58.137741 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-f6630977-eec7-4646-a572-0f1e3b3c052d None None] MFA Rules not processed for user `680d62a67ae2456f8c129e26728c3297`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:18:58.138922 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-f6630977-eec7-4646-a572-0f1e3b3c052d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:18:58.147504 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1213/2422] 10.222.0.18 () {58 vars in 1018 bytes} [Sat Aug 6 22:18:58 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:18:58.158366 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4960d771-ea0f-47c1-a389-46a40dfd3eb7 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:18:58.160133 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-4960d771-ea0f-47c1-a389-46a40dfd3eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:18:58.176713 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4960d771-ea0f-47c1-a389-46a40dfd3eb7 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:18:58.181815 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfc8895-ff67-46b3-b0a6-38a83b6853cb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '680d62a67ae2456f8c129e26728c3297', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:18:58.182426 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfc8895-ff67-46b3-b0a6-38a83b6853cb service placement] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:18:58.182703 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfc8895-ff67-46b3-b0a6-38a83b6853cb service placement] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:18:58.182971 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfc8895-ff67-46b3-b0a6-38a83b6853cb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:18:58.194527 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfc8895-ff67-46b3-b0a6-38a83b6853cb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:18:58.196180 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfc8895-ff67-46b3-b0a6-38a83b6853cb service placement] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:18:58.207703 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1210/2423] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:18:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:19:02.120071 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6630977-eec7-4646-a572-0f1e3b3c052d None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:19:02.122126 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-f6630977-eec7-4646-a572-0f1e3b3c052d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:19:02.137239 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6630977-eec7-4646-a572-0f1e3b3c052d None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:19:02.141345 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13a194c-33af-496a-bf5f-dca8fc55542c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:19:02.141964 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13a194c-33af-496a-bf5f-dca8fc55542c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:19:02.142266 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13a194c-33af-496a-bf5f-dca8fc55542c admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:19:02.142533 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13a194c-33af-496a-bf5f-dca8fc55542c admin admin] PATH_INFO: `/v3/users/7a851cc02ef54ebc9101964bb5c0d9d7` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:19:02.143633 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13a194c-33af-496a-bf5f-dca8fc55542c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a851cc02ef54ebc9101964bb5c0d9d7)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:19:02.145588 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13a194c-33af-496a-bf5f-dca8fc55542c admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:19:02.159134 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:19:02.406302 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-e13a194c-33af-496a-bf5f-dca8fc55542c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a851cc02ef54ebc9101964bb5c0d9d7'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:19:02.418350 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-e13a194c-33af-496a-bf5f-dca8fc55542c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a851cc02ef54ebc9101964bb5c0d9d7'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:19:02.425182 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-e13a194c-33af-496a-bf5f-dca8fc55542c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a851cc02ef54ebc9101964bb5c0d9d7'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:19:02.425853 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-e13a194c-33af-496a-bf5f-dca8fc55542c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a851cc02ef54ebc9101964bb5c0d9d7'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:19:02.426467 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-e13a194c-33af-496a-bf5f-dca8fc55542c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a851cc02ef54ebc9101964bb5c0d9d7'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:19:02.528249 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1214/2424] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:19:02 2022] DELETE /identity/v3/users/7a851cc02ef54ebc9101964bb5c0d9d7 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:19:02.787246 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfc8895-ff67-46b3-b0a6-38a83b6853cb service placement] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:19:02.798659 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfc8895-ff67-46b3-b0a6-38a83b6853cb service placement] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:19:02.864610 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07650aa1-18b1-4a29-a5e0-9c1d7f9970e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:19:02.865247 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07650aa1-18b1-4a29-a5e0-9c1d7f9970e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:19:02.865525 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07650aa1-18b1-4a29-a5e0-9c1d7f9970e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:19:02.865799 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07650aa1-18b1-4a29-a5e0-9c1d7f9970e2 admin admin] PATH_INFO: `/v3/projects/a375619d804c4675a5b9115aa13759a8` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:19:02.866929 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07650aa1-18b1-4a29-a5e0-9c1d7f9970e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a375619d804c4675a5b9115aa13759a8)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:19:02.868880 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07650aa1-18b1-4a29-a5e0-9c1d7f9970e2 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:19:02.877713 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:19:02.979891 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-07650aa1-18b1-4a29-a5e0-9c1d7f9970e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a375619d804c4675a5b9115aa13759a8', 'request_id': 'req-07650aa1-18b1-4a29-a5e0-9c1d7f9970e2'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:19:03.054410 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-07650aa1-18b1-4a29-a5e0-9c1d7f9970e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a375619d804c4675a5b9115aa13759a8', 'request_id': 'req-07650aa1-18b1-4a29-a5e0-9c1d7f9970e2'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:19:03.061041 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-07650aa1-18b1-4a29-a5e0-9c1d7f9970e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a375619d804c4675a5b9115aa13759a8', 'request_id': 'req-07650aa1-18b1-4a29-a5e0-9c1d7f9970e2'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:19:03.072132 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-07650aa1-18b1-4a29-a5e0-9c1d7f9970e2 admin admin] The token cache is being invalidate because project a375619d804c4675a5b9115aa13759a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:19:03.072441 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-07650aa1-18b1-4a29-a5e0-9c1d7f9970e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:19:03.073110 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-07650aa1-18b1-4a29-a5e0-9c1d7f9970e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:19:03.074904 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1211/2425] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:19:02 2022] DELETE /identity/v3/projects/a375619d804c4675a5b9115aa13759a8 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:19:03.078894 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13a194c-33af-496a-bf5f-dca8fc55542c admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:19:03.080359 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-e13a194c-33af-496a-bf5f-dca8fc55542c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:19:03.093418 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13a194c-33af-496a-bf5f-dca8fc55542c admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:19:03.149598 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050b7d0d-efb2-447e-a7b4-6faf0f5b84dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:19:03.150233 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b7d0d-efb2-447e-a7b4-6faf0f5b84dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:19:03.150488 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b7d0d-efb2-447e-a7b4-6faf0f5b84dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:19:03.150763 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b7d0d-efb2-447e-a7b4-6faf0f5b84dd admin admin] PATH_INFO: `/v3/users/d960b33daee341d98c3e41fb690adbb3` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:19:03.151881 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050b7d0d-efb2-447e-a7b4-6faf0f5b84dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=d960b33daee341d98c3e41fb690adbb3)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:19:03.154381 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050b7d0d-efb2-447e-a7b4-6faf0f5b84dd admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:19:03.343780 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-050b7d0d-efb2-447e-a7b4-6faf0f5b84dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd960b33daee341d98c3e41fb690adbb3'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:19:03.358179 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-050b7d0d-efb2-447e-a7b4-6faf0f5b84dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd960b33daee341d98c3e41fb690adbb3'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:19:03.365145 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-050b7d0d-efb2-447e-a7b4-6faf0f5b84dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd960b33daee341d98c3e41fb690adbb3'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:19:03.365856 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-050b7d0d-efb2-447e-a7b4-6faf0f5b84dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd960b33daee341d98c3e41fb690adbb3'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:19:03.366482 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-050b7d0d-efb2-447e-a7b4-6faf0f5b84dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd960b33daee341d98c3e41fb690adbb3'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:19:03.461110 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1215/2426] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:19:03 2022] DELETE /identity/v3/users/d960b33daee341d98c3e41fb690adbb3 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:19:03.724547 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07650aa1-18b1-4a29-a5e0-9c1d7f9970e2 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:19:03.729422 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-07650aa1-18b1-4a29-a5e0-9c1d7f9970e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:19:03.739324 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07650aa1-18b1-4a29-a5e0-9c1d7f9970e2 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:19:03.797301 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dbc86a-4c24-44d3-876c-c23e4a57863b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:19:03.797930 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dbc86a-4c24-44d3-876c-c23e4a57863b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:19:03.798195 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dbc86a-4c24-44d3-876c-c23e4a57863b admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:19:03.798467 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dbc86a-4c24-44d3-876c-c23e4a57863b admin admin] PATH_INFO: `/v3/projects/56add7eeecdc45ddb0f142a6ab36754c` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:19:03.799577 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dbc86a-4c24-44d3-876c-c23e4a57863b admin admin] RBAC: Authorizing `identity:delete_project(project_id=56add7eeecdc45ddb0f142a6ab36754c)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:19:03.801595 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dbc86a-4c24-44d3-876c-c23e4a57863b admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:19:03.906477 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-85dbc86a-4c24-44d3-876c-c23e4a57863b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56add7eeecdc45ddb0f142a6ab36754c', 'request_id': 'req-85dbc86a-4c24-44d3-876c-c23e4a57863b'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:19:03.986121 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-85dbc86a-4c24-44d3-876c-c23e4a57863b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56add7eeecdc45ddb0f142a6ab36754c', 'request_id': 'req-85dbc86a-4c24-44d3-876c-c23e4a57863b'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:19:03.991812 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-85dbc86a-4c24-44d3-876c-c23e4a57863b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56add7eeecdc45ddb0f142a6ab36754c', 'request_id': 'req-85dbc86a-4c24-44d3-876c-c23e4a57863b'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:19:04.004025 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-85dbc86a-4c24-44d3-876c-c23e4a57863b admin admin] The token cache is being invalidate because project 56add7eeecdc45ddb0f142a6ab36754c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:19:04.010154 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-85dbc86a-4c24-44d3-876c-c23e4a57863b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:19:04.010154 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-85dbc86a-4c24-44d3-876c-c23e4a57863b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:19:04.010154 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1212/2427] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:19:03 2022] DELETE /identity/v3/projects/56add7eeecdc45ddb0f142a6ab36754c => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:19:13.457229 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=75,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:19:14.000762 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=72,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:20:00.899504 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050b7d0d-efb2-447e-a7b4-6faf0f5b84dd admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:20:00.901206 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-050b7d0d-efb2-447e-a7b4-6faf0f5b84dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:20:01.004930 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050b7d0d-efb2-447e-a7b4-6faf0f5b84dd admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:20:01.064156 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2321b022-9084-47d0-a2bf-a40035d2e8a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:20:01.064830 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2321b022-9084-47d0-a2bf-a40035d2e8a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:01.065104 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2321b022-9084-47d0-a2bf-a40035d2e8a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:01.065371 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2321b022-9084-47d0-a2bf-a40035d2e8a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:01.067119 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-2321b022-9084-47d0-a2bf-a40035d2e8a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:20:01.135474 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2321b022-9084-47d0-a2bf-a40035d2e8a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:20:01.136938 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2321b022-9084-47d0-a2bf-a40035d2e8a6 service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:20:01.147125 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1216/2428] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:20:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 22:20:08.312375 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dbc86a-4c24-44d3-876c-c23e4a57863b admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:20:08.314575 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-85dbc86a-4c24-44d3-876c-c23e4a57863b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:20:08.328520 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dbc86a-4c24-44d3-876c-c23e4a57863b admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:20:08.333351 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6c9760-57ec-4af6-ae5b-4f15f3b3c406 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:20:08.333969 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6c9760-57ec-4af6-ae5b-4f15f3b3c406 service neutron] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:08.334249 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6c9760-57ec-4af6-ae5b-4f15f3b3c406 service neutron] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:08.334572 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6c9760-57ec-4af6-ae5b-4f15f3b3c406 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:08.336497 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-ef6c9760-57ec-4af6-ae5b-4f15f3b3c406 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:20:08.350502 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6c9760-57ec-4af6-ae5b-4f15f3b3c406 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:20:08.351955 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6c9760-57ec-4af6-ae5b-4f15f3b3c406 service neutron] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:20:08.367071 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1213/2429] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:20:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:20:08.973412 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff3d30f-a0d7-46d3-8706-a48db3cb652a None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:08.973921 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff3d30f-a0d7-46d3-8706-a48db3cb652a None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:08.974379 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff3d30f-a0d7-46d3-8706-a48db3cb652a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:09.019045 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-7ff3d30f-a0d7-46d3-8706-a48db3cb652a None None] MFA Rules not processed for user `d1b26ba592724dd389584bb800d877a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:20:09.021391 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-7ff3d30f-a0d7-46d3-8706-a48db3cb652a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:20:09.136929 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:20:09.160440 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1217/2430] 10.222.0.18 () {58 vars in 1009 bytes} [Sat Aug 6 22:20:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:20:09.324796 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6c9760-57ec-4af6-ae5b-4f15f3b3c406 service neutron] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:20:09.327022 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-ef6c9760-57ec-4af6-ae5b-4f15f3b3c406 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:20:09.431122 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6c9760-57ec-4af6-ae5b-4f15f3b3c406 service neutron] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:20:09.450371 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55606d44-e17e-49e4-bf89-42d13157a226 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7fc9da7505744608dfd3bbd46a0a9d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:20:09.452276 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55606d44-e17e-49e4-bf89-42d13157a226 service swift] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:09.453406 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55606d44-e17e-49e4-bf89-42d13157a226 service swift] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:09.453872 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55606d44-e17e-49e4-bf89-42d13157a226 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:09.480344 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55606d44-e17e-49e4-bf89-42d13157a226 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:20:09.482406 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55606d44-e17e-49e4-bf89-42d13157a226 service swift] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:20:09.513723 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1214/2431] 10.222.0.18 () {60 vars in 1384 bytes} [Sat Aug 6 22:20:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 22:20:10.382651 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff3d30f-a0d7-46d3-8706-a48db3cb652a None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:20:10.385121 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-7ff3d30f-a0d7-46d3-8706-a48db3cb652a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:20:10.412581 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff3d30f-a0d7-46d3-8706-a48db3cb652a None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:20:10.449272 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c35f2f-791d-4a42-bd84-909b37c2bd73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:20:10.449791 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c35f2f-791d-4a42-bd84-909b37c2bd73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:10.450074 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c35f2f-791d-4a42-bd84-909b37c2bd73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:10.450348 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c35f2f-791d-4a42-bd84-909b37c2bd73 admin admin] PATH_INFO: `/v3/users/9469cd012c2549288ba403902b667295` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:10.452705 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c35f2f-791d-4a42-bd84-909b37c2bd73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9469cd012c2549288ba403902b667295)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:20:10.458994 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c35f2f-791d-4a42-bd84-909b37c2bd73 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:20:10.727980 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-a0c35f2f-791d-4a42-bd84-909b37c2bd73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9469cd012c2549288ba403902b667295'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:20:10.749721 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-a0c35f2f-791d-4a42-bd84-909b37c2bd73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9469cd012c2549288ba403902b667295'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:20:10.763331 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-a0c35f2f-791d-4a42-bd84-909b37c2bd73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9469cd012c2549288ba403902b667295'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:20:10.764085 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-a0c35f2f-791d-4a42-bd84-909b37c2bd73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9469cd012c2549288ba403902b667295'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:20:10.764784 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-a0c35f2f-791d-4a42-bd84-909b37c2bd73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9469cd012c2549288ba403902b667295'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:20:10.888249 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1218/2432] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:20:10 2022] DELETE /identity/v3/users/9469cd012c2549288ba403902b667295 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:20:11.439597 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55606d44-e17e-49e4-bf89-42d13157a226 service swift] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:20:11.459936 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55606d44-e17e-49e4-bf89-42d13157a226 service swift] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:20:11.575279 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27684eda-2da1-47dd-a834-e578b75d1209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:20:11.575929 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27684eda-2da1-47dd-a834-e578b75d1209 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:11.576221 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27684eda-2da1-47dd-a834-e578b75d1209 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:11.576859 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27684eda-2da1-47dd-a834-e578b75d1209 admin admin] PATH_INFO: `/v3/projects/6354597f8edf4ebc9539a33314e6bc06` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:11.578179 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27684eda-2da1-47dd-a834-e578b75d1209 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6354597f8edf4ebc9539a33314e6bc06)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:20:11.580884 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27684eda-2da1-47dd-a834-e578b75d1209 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:20:11.803200 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-27684eda-2da1-47dd-a834-e578b75d1209 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6354597f8edf4ebc9539a33314e6bc06', 'request_id': 'req-27684eda-2da1-47dd-a834-e578b75d1209'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:20:11.978357 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-27684eda-2da1-47dd-a834-e578b75d1209 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6354597f8edf4ebc9539a33314e6bc06', 'request_id': 'req-27684eda-2da1-47dd-a834-e578b75d1209'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:20:11.994804 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-27684eda-2da1-47dd-a834-e578b75d1209 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6354597f8edf4ebc9539a33314e6bc06', 'request_id': 'req-27684eda-2da1-47dd-a834-e578b75d1209'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:20:12.021481 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-27684eda-2da1-47dd-a834-e578b75d1209 admin admin] The token cache is being invalidate because project 6354597f8edf4ebc9539a33314e6bc06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:20:12.021770 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-27684eda-2da1-47dd-a834-e578b75d1209 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:20:12.023478 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-27684eda-2da1-47dd-a834-e578b75d1209 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:20:12.025773 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1215/2433] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:20:11 2022] DELETE /identity/v3/projects/6354597f8edf4ebc9539a33314e6bc06 => generated 0 bytes in 587 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:20:12.029668 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c35f2f-791d-4a42-bd84-909b37c2bd73 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:20:12.031937 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-a0c35f2f-791d-4a42-bd84-909b37c2bd73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:20:12.059485 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c35f2f-791d-4a42-bd84-909b37c2bd73 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:20:12.217810 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5416768-d119-4d34-82df-e538363a6e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:20:12.219145 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5416768-d119-4d34-82df-e538363a6e45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:12.221235 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5416768-d119-4d34-82df-e538363a6e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:12.228034 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5416768-d119-4d34-82df-e538363a6e45 admin admin] PATH_INFO: `/v3/users/fbbe8c7337ac4697b4a8dfbc0569943c` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:12.230103 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5416768-d119-4d34-82df-e538363a6e45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbbe8c7337ac4697b4a8dfbc0569943c)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:20:12.233908 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5416768-d119-4d34-82df-e538363a6e45 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:20:12.575738 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-c5416768-d119-4d34-82df-e538363a6e45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbbe8c7337ac4697b4a8dfbc0569943c'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:20:12.589443 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-c5416768-d119-4d34-82df-e538363a6e45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbbe8c7337ac4697b4a8dfbc0569943c'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:20:12.599773 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-c5416768-d119-4d34-82df-e538363a6e45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbbe8c7337ac4697b4a8dfbc0569943c'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:20:12.600961 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-c5416768-d119-4d34-82df-e538363a6e45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbbe8c7337ac4697b4a8dfbc0569943c'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:20:12.601675 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-c5416768-d119-4d34-82df-e538363a6e45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbbe8c7337ac4697b4a8dfbc0569943c'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:20:12.603537 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:20:12.708557 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1219/2434] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:20:12 2022] DELETE /identity/v3/users/fbbe8c7337ac4697b4a8dfbc0569943c => generated 0 bytes in 678 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:20:13.077613 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27684eda-2da1-47dd-a834-e578b75d1209 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:20:13.079648 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-27684eda-2da1-47dd-a834-e578b75d1209 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:20:13.108180 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27684eda-2da1-47dd-a834-e578b75d1209 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:20:13.118393 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:20:13.208006 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b8c7b0-1f22-485c-91ec-a10a26dcd3cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:20:13.209359 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b8c7b0-1f22-485c-91ec-a10a26dcd3cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:13.209957 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b8c7b0-1f22-485c-91ec-a10a26dcd3cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:13.210524 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b8c7b0-1f22-485c-91ec-a10a26dcd3cf admin admin] PATH_INFO: `/v3/projects/e656860e14b044deb38ed079fcb5380a` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:13.211958 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b8c7b0-1f22-485c-91ec-a10a26dcd3cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=e656860e14b044deb38ed079fcb5380a)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:20:13.214495 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b8c7b0-1f22-485c-91ec-a10a26dcd3cf admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:20:13.391331 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-08b8c7b0-1f22-485c-91ec-a10a26dcd3cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e656860e14b044deb38ed079fcb5380a', 'request_id': 'req-08b8c7b0-1f22-485c-91ec-a10a26dcd3cf'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:20:13.942733 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-08b8c7b0-1f22-485c-91ec-a10a26dcd3cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e656860e14b044deb38ed079fcb5380a', 'request_id': 'req-08b8c7b0-1f22-485c-91ec-a10a26dcd3cf'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:20:13.948759 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-08b8c7b0-1f22-485c-91ec-a10a26dcd3cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e656860e14b044deb38ed079fcb5380a', 'request_id': 'req-08b8c7b0-1f22-485c-91ec-a10a26dcd3cf'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:20:13.961118 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-08b8c7b0-1f22-485c-91ec-a10a26dcd3cf admin admin] The token cache is being invalidate because project e656860e14b044deb38ed079fcb5380a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:20:13.961396 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-08b8c7b0-1f22-485c-91ec-a10a26dcd3cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:20:13.962028 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-08b8c7b0-1f22-485c-91ec-a10a26dcd3cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:20:13.963506 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1216/2435] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:20:13 2022] DELETE /identity/v3/projects/e656860e14b044deb38ed079fcb5380a => generated 0 bytes in 886 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:20:15.043133 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5416768-d119-4d34-82df-e538363a6e45 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:20:15.044655 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-c5416768-d119-4d34-82df-e538363a6e45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:20:15.068123 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5416768-d119-4d34-82df-e538363a6e45 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:20:15.146076 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73111507-1938-4499-bbd1-7d2066537c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:20:15.146772 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73111507-1938-4499-bbd1-7d2066537c54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:15.147062 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73111507-1938-4499-bbd1-7d2066537c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:15.147336 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73111507-1938-4499-bbd1-7d2066537c54 admin admin] PATH_INFO: `/v3/users/700f40b4567a4bcdbb2194861f4497d5` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:15.148531 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73111507-1938-4499-bbd1-7d2066537c54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=700f40b4567a4bcdbb2194861f4497d5)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:20:15.150551 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73111507-1938-4499-bbd1-7d2066537c54 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:20:15.406624 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-73111507-1938-4499-bbd1-7d2066537c54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '700f40b4567a4bcdbb2194861f4497d5'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:20:15.424523 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-73111507-1938-4499-bbd1-7d2066537c54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '700f40b4567a4bcdbb2194861f4497d5'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:20:15.433554 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-73111507-1938-4499-bbd1-7d2066537c54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '700f40b4567a4bcdbb2194861f4497d5'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:20:15.434340 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-73111507-1938-4499-bbd1-7d2066537c54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '700f40b4567a4bcdbb2194861f4497d5'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:20:15.435431 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-73111507-1938-4499-bbd1-7d2066537c54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '700f40b4567a4bcdbb2194861f4497d5'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:20:15.545997 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1220/2436] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:20:15 2022] DELETE /identity/v3/users/700f40b4567a4bcdbb2194861f4497d5 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:20:15.845515 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6828e6e-86d7-43c7-a6e9-7c166c0af10c None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:15.846056 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6828e6e-86d7-43c7-a6e9-7c166c0af10c None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:15.846056 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6828e6e-86d7-43c7-a6e9-7c166c0af10c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:15.884473 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-a6828e6e-86d7-43c7-a6e9-7c166c0af10c None None] MFA Rules not processed for user `0d6e35038fed4b44b986902a5ef55ce8`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:20:15.885853 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-a6828e6e-86d7-43c7-a6e9-7c166c0af10c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:20:15.937031 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73111507-1938-4499-bbd1-7d2066537c54 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:20:15.940166 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-73111507-1938-4499-bbd1-7d2066537c54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:20:15.990153 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1217/2437] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:20:15 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:20:16.043466 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73111507-1938-4499-bbd1-7d2066537c54 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:20:16.048558 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da13c6b-fb6a-4693-8044-d49a1cdef2b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:20:16.049220 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da13c6b-fb6a-4693-8044-d49a1cdef2b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:16.049690 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da13c6b-fb6a-4693-8044-d49a1cdef2b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:16.050054 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da13c6b-fb6a-4693-8044-d49a1cdef2b5 admin admin] PATH_INFO: `/v3/projects/480896a54dab43c8980c8b05c9a793e7` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:16.051329 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da13c6b-fb6a-4693-8044-d49a1cdef2b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=480896a54dab43c8980c8b05c9a793e7)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:20:16.053308 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da13c6b-fb6a-4693-8044-d49a1cdef2b5 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:20:16.089159 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:20:16.225483 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-1da13c6b-fb6a-4693-8044-d49a1cdef2b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '480896a54dab43c8980c8b05c9a793e7', 'request_id': 'req-1da13c6b-fb6a-4693-8044-d49a1cdef2b5'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:20:16.321197 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-1da13c6b-fb6a-4693-8044-d49a1cdef2b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '480896a54dab43c8980c8b05c9a793e7', 'request_id': 'req-1da13c6b-fb6a-4693-8044-d49a1cdef2b5'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:20:16.338736 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-1da13c6b-fb6a-4693-8044-d49a1cdef2b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '480896a54dab43c8980c8b05c9a793e7', 'request_id': 'req-1da13c6b-fb6a-4693-8044-d49a1cdef2b5'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:20:16.385004 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-1da13c6b-fb6a-4693-8044-d49a1cdef2b5 admin admin] The token cache is being invalidate because project 480896a54dab43c8980c8b05c9a793e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:20:16.385358 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-1da13c6b-fb6a-4693-8044-d49a1cdef2b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:20:16.386472 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-1da13c6b-fb6a-4693-8044-d49a1cdef2b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:20:16.388218 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1221/2438] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:20:15 2022] DELETE /identity/v3/projects/480896a54dab43c8980c8b05c9a793e7 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:20:16.393045 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd325b2-6571-4ba9-9e6a-8d3461d53bc4 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:16.393449 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd325b2-6571-4ba9-9e6a-8d3461d53bc4 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:16.393824 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd325b2-6571-4ba9-9e6a-8d3461d53bc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:16.469136 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-4fd325b2-6571-4ba9-9e6a-8d3461d53bc4 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:20:16.470594 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-4fd325b2-6571-4ba9-9e6a-8d3461d53bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:20:16.480292 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:20:16.557495 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1218/2439] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:20:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:20:16.564838 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f742ca-123c-4332-9b6f-7b8f95ae232d None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:16.565816 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f742ca-123c-4332-9b6f-7b8f95ae232d None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:16.566157 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f742ca-123c-4332-9b6f-7b8f95ae232d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:16.589877 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-85f742ca-123c-4332-9b6f-7b8f95ae232d None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:20:16.591213 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-85f742ca-123c-4332-9b6f-7b8f95ae232d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:20:16.600062 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1222/2440] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:20:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 22:20:16.604879 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd325b2-6571-4ba9-9e6a-8d3461d53bc4 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:20:16.615612 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd325b2-6571-4ba9-9e6a-8d3461d53bc4 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:20:16.621643 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35992e62-56c4-4b47-8d1e-03a5d5aa5db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:20:16.622351 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35992e62-56c4-4b47-8d1e-03a5d5aa5db7 admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:16.622692 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35992e62-56c4-4b47-8d1e-03a5d5aa5db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:16.623029 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35992e62-56c4-4b47-8d1e-03a5d5aa5db7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:16.623755 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35992e62-56c4-4b47-8d1e-03a5d5aa5db7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 22:20:16.626389 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35992e62-56c4-4b47-8d1e-03a5d5aa5db7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:20:16.628296 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35992e62-56c4-4b47-8d1e-03a5d5aa5db7 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:20:16.641377 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1219/2441] 10.222.0.18 () {58 vars in 1153 bytes} [Sat Aug 6 22:20:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 22:20:16.645649 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f742ca-123c-4332-9b6f-7b8f95ae232d None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:20:16.655614 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f742ca-123c-4332-9b6f-7b8f95ae232d None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:20:16.666287 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504a2816-36fe-438c-b8de-560cb26fa728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:20:16.667086 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504a2816-36fe-438c-b8de-560cb26fa728 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:16.667448 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504a2816-36fe-438c-b8de-560cb26fa728 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:16.667788 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504a2816-36fe-438c-b8de-560cb26fa728 admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:16.668783 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504a2816-36fe-438c-b8de-560cb26fa728 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:20:16.670948 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504a2816-36fe-438c-b8de-560cb26fa728 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:20:16.826414 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1223/2442] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:20:16 2022] POST /identity/v3/projects => generated 399 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 22:20:16.831380 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35992e62-56c4-4b47-8d1e-03a5d5aa5db7 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:20:16.845372 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35992e62-56c4-4b47-8d1e-03a5d5aa5db7 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:20:16.985919 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1e7f96-0d83-4a58-9bf8-cc263d0e0f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:20:16.987837 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1e7f96-0d83-4a58-9bf8-cc263d0e0f80 admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:16.988150 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1e7f96-0d83-4a58-9bf8-cc263d0e0f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:16.989953 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1e7f96-0d83-4a58-9bf8-cc263d0e0f80 admin admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:16.990686 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1e7f96-0d83-4a58-9bf8-cc263d0e0f80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:20:16.999574 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1e7f96-0d83-4a58-9bf8-cc263d0e0f80 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:20:17.124545 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1220/2443] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:20:16 2022] POST /identity/v3/users => generated 300 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:20:17.129153 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504a2816-36fe-438c-b8de-560cb26fa728 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:20:17.139831 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504a2816-36fe-438c-b8de-560cb26fa728 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:20:17.152494 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de534905-4916-47f1-bcf5-d26ce4a1d71c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:20:17.153054 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de534905-4916-47f1-bcf5-d26ce4a1d71c admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:17.153330 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de534905-4916-47f1-bcf5-d26ce4a1d71c admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:17.153622 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de534905-4916-47f1-bcf5-d26ce4a1d71c admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:17.154237 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de534905-4916-47f1-bcf5-d26ce4a1d71c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:20:17.155855 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de534905-4916-47f1-bcf5-d26ce4a1d71c admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:20:17.172513 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1224/2444] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:20:17 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:20:17.176747 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1e7f96-0d83-4a58-9bf8-cc263d0e0f80 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:20:17.189342 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1e7f96-0d83-4a58-9bf8-cc263d0e0f80 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:20:17.196861 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f29725-4a73-42ce-9126-c6ec525adcfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:20:17.197976 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f29725-4a73-42ce-9126-c6ec525adcfb admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:17.198428 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f29725-4a73-42ce-9126-c6ec525adcfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:17.198801 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f29725-4a73-42ce-9126-c6ec525adcfb admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:17.199591 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f29725-4a73-42ce-9126-c6ec525adcfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:20:17.201953 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f29725-4a73-42ce-9126-c6ec525adcfb admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:20:17.219300 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1221/2445] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:20:17 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:20:17.224914 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de534905-4916-47f1-bcf5-d26ce4a1d71c admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:20:17.238682 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de534905-4916-47f1-bcf5-d26ce4a1d71c admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:20:17.245260 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf0d28a-ebe9-4a1c-a765-a5f142e9d899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:20:17.246122 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf0d28a-ebe9-4a1c-a765-a5f142e9d899 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:17.246472 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf0d28a-ebe9-4a1c-a765-a5f142e9d899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:17.246849 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf0d28a-ebe9-4a1c-a765-a5f142e9d899 admin admin] PATH_INFO: `/v3/projects/0bfceb48d37f43f299f51aa439135ac9/users/b2e07d9a418442948031c95bc075c361/roles/56e4a96094e9434cbbafb22579c034f5` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:17.263882 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf0d28a-ebe9-4a1c-a765-a5f142e9d899 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bfceb48d37f43f299f51aa439135ac9, user_id=b2e07d9a418442948031c95bc075c361, role_id=56e4a96094e9434cbbafb22579c034f5)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:20:17.266576 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf0d28a-ebe9-4a1c-a765-a5f142e9d899 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:20:17.407498 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1225/2446] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:20:17 2022] PUT /identity/v3/projects/0bfceb48d37f43f299f51aa439135ac9/users/b2e07d9a418442948031c95bc075c361/roles/56e4a96094e9434cbbafb22579c034f5 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:20:17.410195 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2388cf-f256-403d-8a15-0564ced8a483 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:17.410498 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2388cf-f256-403d-8a15-0564ced8a483 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:17.410758 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2388cf-f256-403d-8a15-0564ced8a483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:17.441541 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-2b2388cf-f256-403d-8a15-0564ced8a483 None None] MFA Rules not processed for user `b2e07d9a418442948031c95bc075c361`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:20:17.441984 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-2b2388cf-f256-403d-8a15-0564ced8a483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:20:17.515436 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1222/2447] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:20:17 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:20:17.520961 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf0d28a-ebe9-4a1c-a765-a5f142e9d899 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:20:17.603529 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf0d28a-ebe9-4a1c-a765-a5f142e9d899 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:20:17.608485 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e50a67-4f9c-456f-b74b-e037a3ab14c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:20:17.609105 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e50a67-4f9c-456f-b74b-e037a3ab14c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:17.609382 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e50a67-4f9c-456f-b74b-e037a3ab14c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:17.609654 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e50a67-4f9c-456f-b74b-e037a3ab14c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:17.610299 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e50a67-4f9c-456f-b74b-e037a3ab14c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:20:17.613288 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e50a67-4f9c-456f-b74b-e037a3ab14c8 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:20:18.187910 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1226/2448] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:20:17 2022] POST /identity/v3/projects => generated 399 bytes in 665 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:20:18.191427 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2388cf-f256-403d-8a15-0564ced8a483 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:20:18.202886 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2388cf-f256-403d-8a15-0564ced8a483 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:20:18.207116 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:20:18.267605 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5fc6bb-aa51-4ca6-ae61-7bbdd6cfc1da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:20:18.269090 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5fc6bb-aa51-4ca6-ae61-7bbdd6cfc1da admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:18.269369 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5fc6bb-aa51-4ca6-ae61-7bbdd6cfc1da admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:18.269670 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5fc6bb-aa51-4ca6-ae61-7bbdd6cfc1da admin admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:18.270299 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5fc6bb-aa51-4ca6-ae61-7bbdd6cfc1da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:20:18.272247 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5fc6bb-aa51-4ca6-ae61-7bbdd6cfc1da admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:20:18.353355 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1223/2449] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:20:18 2022] POST /identity/v3/users => generated 306 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:20:18.358359 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e50a67-4f9c-456f-b74b-e037a3ab14c8 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:20:18.371691 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e50a67-4f9c-456f-b74b-e037a3ab14c8 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:20:18.376696 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f590283-48c1-469d-8a66-f7b8efef00bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:20:18.377834 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f590283-48c1-469d-8a66-f7b8efef00bc admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:18.378119 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f590283-48c1-469d-8a66-f7b8efef00bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:18.378395 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f590283-48c1-469d-8a66-f7b8efef00bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:18.379064 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f590283-48c1-469d-8a66-f7b8efef00bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:20:18.380691 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f590283-48c1-469d-8a66-f7b8efef00bc admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:20:18.392464 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1227/2450] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:20:18 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 22:20:18.397743 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5fc6bb-aa51-4ca6-ae61-7bbdd6cfc1da admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:20:18.408619 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5fc6bb-aa51-4ca6-ae61-7bbdd6cfc1da admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:20:18.413593 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deefae89-8eab-426b-97b9-4eaf36ea4161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:20:18.414306 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deefae89-8eab-426b-97b9-4eaf36ea4161 admin admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:18.414587 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deefae89-8eab-426b-97b9-4eaf36ea4161 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:18.414861 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deefae89-8eab-426b-97b9-4eaf36ea4161 admin admin] PATH_INFO: `/v3/projects/059f605e00d64a86b4b931ff3ce2490c/users/6a9a3abeb82246e6834a8022da006ba9/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:18.434593 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deefae89-8eab-426b-97b9-4eaf36ea4161 admin admin] RBAC: Authorizing `identity:create_grant(project_id=059f605e00d64a86b4b931ff3ce2490c, user_id=6a9a3abeb82246e6834a8022da006ba9, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:20:18.441022 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deefae89-8eab-426b-97b9-4eaf36ea4161 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:20:18.546246 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1224/2451] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:20:18 2022] PUT /identity/v3/projects/059f605e00d64a86b4b931ff3ce2490c/users/6a9a3abeb82246e6834a8022da006ba9/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:20:18.550856 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f590283-48c1-469d-8a66-f7b8efef00bc admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:20:18.556295 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:20:18.566157 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f590283-48c1-469d-8a66-f7b8efef00bc admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:20:18.635573 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d407d3-68a3-4261-a3b8-707fa2c950e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:20:18.636206 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d407d3-68a3-4261-a3b8-707fa2c950e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:18.636519 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d407d3-68a3-4261-a3b8-707fa2c950e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:18.636798 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d407d3-68a3-4261-a3b8-707fa2c950e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:18.637451 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d407d3-68a3-4261-a3b8-707fa2c950e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:20:18.639099 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d407d3-68a3-4261-a3b8-707fa2c950e4 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:20:18.650164 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1228/2452] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:20:18 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 22:20:18.654583 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deefae89-8eab-426b-97b9-4eaf36ea4161 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:20:18.671370 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deefae89-8eab-426b-97b9-4eaf36ea4161 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:20:18.677110 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb26dc6-0766-4c43-a6d5-f87e711a19c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:20:18.677870 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb26dc6-0766-4c43-a6d5-f87e711a19c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:18.678219 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb26dc6-0766-4c43-a6d5-f87e711a19c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:18.678561 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb26dc6-0766-4c43-a6d5-f87e711a19c8 admin admin] PATH_INFO: `/v3/projects/059f605e00d64a86b4b931ff3ce2490c/users/6a9a3abeb82246e6834a8022da006ba9/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:18.680602 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb26dc6-0766-4c43-a6d5-f87e711a19c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=059f605e00d64a86b4b931ff3ce2490c, user_id=6a9a3abeb82246e6834a8022da006ba9, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:20:18.683292 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb26dc6-0766-4c43-a6d5-f87e711a19c8 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:20:18.696367 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1225/2453] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:20:18 2022] PUT /identity/v3/projects/059f605e00d64a86b4b931ff3ce2490c/users/6a9a3abeb82246e6834a8022da006ba9/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:20:18.708108 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af90f69-e564-4d78-8bcb-dcb3f7d2880f None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:18.708432 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af90f69-e564-4d78-8bcb-dcb3f7d2880f None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:18.708738 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af90f69-e564-4d78-8bcb-dcb3f7d2880f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:18.741736 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-6af90f69-e564-4d78-8bcb-dcb3f7d2880f None None] MFA Rules not processed for user `6a9a3abeb82246e6834a8022da006ba9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:20:18.743121 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-6af90f69-e564-4d78-8bcb-dcb3f7d2880f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:20:18.837078 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1229/2454] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:20:18 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:20:18.853930 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb26dc6-0766-4c43-a6d5-f87e711a19c8 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:20:18.855885 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-ddb26dc6-0766-4c43-a6d5-f87e711a19c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:20:18.950898 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb26dc6-0766-4c43-a6d5-f87e711a19c8 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:20:18.955209 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd34c05-ec2e-4757-b13a-fb6c1379c6c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa71c71de263478da234f3c5716a6e5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:20:18.956761 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd34c05-ec2e-4757-b13a-fb6c1379c6c8 service cinder] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:18.957048 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd34c05-ec2e-4757-b13a-fb6c1379c6c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:18.957375 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd34c05-ec2e-4757-b13a-fb6c1379c6c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:18.969288 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd34c05-ec2e-4757-b13a-fb6c1379c6c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:20:18.970485 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd34c05-ec2e-4757-b13a-fb6c1379c6c8 service cinder] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:20:18.984151 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1226/2455] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:20:18 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:20:19.291082 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af90f69-e564-4d78-8bcb-dcb3f7d2880f None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:20:19.301741 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af90f69-e564-4d78-8bcb-dcb3f7d2880f None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:20:19.305695 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b3b3b1-f6e8-40ad-a82a-0984a26f13a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa71c71de263478da234f3c5716a6e5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:20:19.306587 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b3b3b1-f6e8-40ad-a82a-0984a26f13a4 service cinder] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:19.307096 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b3b3b1-f6e8-40ad-a82a-0984a26f13a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:19.307473 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b3b3b1-f6e8-40ad-a82a-0984a26f13a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:19.382427 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b3b3b1-f6e8-40ad-a82a-0984a26f13a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:20:19.384860 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b3b3b1-f6e8-40ad-a82a-0984a26f13a4 service cinder] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:20:19.397052 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1230/2456] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:20:19 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:20:19.405521 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd34c05-ec2e-4757-b13a-fb6c1379c6c8 service cinder] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:20:19.407558 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-0cd34c05-ec2e-4757-b13a-fb6c1379c6c8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:20:19.469752 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:20:19.488004 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd34c05-ec2e-4757-b13a-fb6c1379c6c8 service cinder] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:20:19.493246 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffa60ef-1ed0-41c6-9e19-89aff7a21129 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:20:19.494127 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffa60ef-1ed0-41c6-9e19-89aff7a21129 service neutron] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:19.494694 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffa60ef-1ed0-41c6-9e19-89aff7a21129 service neutron] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:19.495180 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffa60ef-1ed0-41c6-9e19-89aff7a21129 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:19.497019 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-2ffa60ef-1ed0-41c6-9e19-89aff7a21129 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:20:19.649025 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffa60ef-1ed0-41c6-9e19-89aff7a21129 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:20:19.650670 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffa60ef-1ed0-41c6-9e19-89aff7a21129 service neutron] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:20:19.681902 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1227/2457] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:20:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:20:20.292461 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b3b3b1-f6e8-40ad-a82a-0984a26f13a4 service cinder] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:20:20.303437 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b3b3b1-f6e8-40ad-a82a-0984a26f13a4 service cinder] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:20:20.307632 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddba1383-4419-4529-8250-a623ca28be61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:20:20.308543 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddba1383-4419-4529-8250-a623ca28be61 service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:20.308888 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddba1383-4419-4529-8250-a623ca28be61 service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:20.309374 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddba1383-4419-4529-8250-a623ca28be61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:20.311268 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-ddba1383-4419-4529-8250-a623ca28be61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:20:20.333504 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddba1383-4419-4529-8250-a623ca28be61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:20:20.335106 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddba1383-4419-4529-8250-a623ca28be61 service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:20:20.357068 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1231/2458] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:20:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:20:25.809572 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffa60ef-1ed0-41c6-9e19-89aff7a21129 service neutron] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:20:25.875003 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffa60ef-1ed0-41c6-9e19-89aff7a21129 service neutron] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:20:25.880355 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64889114-4fe0-46cb-a5cc-6abe61ca227f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:20:25.881210 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64889114-4fe0-46cb-a5cc-6abe61ca227f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:25.881486 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64889114-4fe0-46cb-a5cc-6abe61ca227f admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:25.881754 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64889114-4fe0-46cb-a5cc-6abe61ca227f admin admin] PATH_INFO: `/v3/users/b2e07d9a418442948031c95bc075c361` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:25.882877 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64889114-4fe0-46cb-a5cc-6abe61ca227f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2e07d9a418442948031c95bc075c361)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:20:25.884823 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64889114-4fe0-46cb-a5cc-6abe61ca227f admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:20:26.142863 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-64889114-4fe0-46cb-a5cc-6abe61ca227f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2e07d9a418442948031c95bc075c361'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:20:26.161069 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-64889114-4fe0-46cb-a5cc-6abe61ca227f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2e07d9a418442948031c95bc075c361'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:20:26.162975 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:20:26.280503 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-64889114-4fe0-46cb-a5cc-6abe61ca227f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2e07d9a418442948031c95bc075c361'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:20:26.288393 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-64889114-4fe0-46cb-a5cc-6abe61ca227f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2e07d9a418442948031c95bc075c361'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:20:26.289150 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-64889114-4fe0-46cb-a5cc-6abe61ca227f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2e07d9a418442948031c95bc075c361'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:20:26.296439 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1228/2459] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:20:25 2022] DELETE /identity/v3/users/b2e07d9a418442948031c95bc075c361 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:20:26.302011 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a07b2d-5291-4d9a-9027-7c6cacdaf766 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:26.303061 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a07b2d-5291-4d9a-9027-7c6cacdaf766 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:26.303661 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a07b2d-5291-4d9a-9027-7c6cacdaf766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:26.322900 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-19a07b2d-5291-4d9a-9027-7c6cacdaf766 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:20:26.325767 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-19a07b2d-5291-4d9a-9027-7c6cacdaf766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:20:26.330547 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:20:26.393685 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1232/2460] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:20:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 22:20:26.404219 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64889114-4fe0-46cb-a5cc-6abe61ca227f admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:20:26.406094 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-64889114-4fe0-46cb-a5cc-6abe61ca227f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:20:26.477091 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64889114-4fe0-46cb-a5cc-6abe61ca227f admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:20:26.480985 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f6c856-0a8a-4a98-8dfe-081584fcc900 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:20:26.481592 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f6c856-0a8a-4a98-8dfe-081584fcc900 service neutron] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:26.481867 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f6c856-0a8a-4a98-8dfe-081584fcc900 service neutron] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:26.482137 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f6c856-0a8a-4a98-8dfe-081584fcc900 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:26.492961 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f6c856-0a8a-4a98-8dfe-081584fcc900 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:20:26.494421 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f6c856-0a8a-4a98-8dfe-081584fcc900 service neutron] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:20:26.504929 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1229/2461] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:20:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:20:26.969631 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a07b2d-5291-4d9a-9027-7c6cacdaf766 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:20:26.971505 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-19a07b2d-5291-4d9a-9027-7c6cacdaf766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:20:26.985165 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a07b2d-5291-4d9a-9027-7c6cacdaf766 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:20:26.990124 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fb9bd2-13d2-426a-98a8-0cc5d7ca7ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:20:26.990753 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb9bd2-13d2-426a-98a8-0cc5d7ca7ee6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:26.991032 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb9bd2-13d2-426a-98a8-0cc5d7ca7ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:26.991306 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb9bd2-13d2-426a-98a8-0cc5d7ca7ee6 admin admin] PATH_INFO: `/v3/projects/0bfceb48d37f43f299f51aa439135ac9` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:26.993035 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fb9bd2-13d2-426a-98a8-0cc5d7ca7ee6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0bfceb48d37f43f299f51aa439135ac9)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:20:26.995927 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fb9bd2-13d2-426a-98a8-0cc5d7ca7ee6 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:20:27.145566 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-83fb9bd2-13d2-426a-98a8-0cc5d7ca7ee6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bfceb48d37f43f299f51aa439135ac9', 'request_id': 'req-83fb9bd2-13d2-426a-98a8-0cc5d7ca7ee6'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:20:27.206325 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-83fb9bd2-13d2-426a-98a8-0cc5d7ca7ee6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bfceb48d37f43f299f51aa439135ac9', 'request_id': 'req-83fb9bd2-13d2-426a-98a8-0cc5d7ca7ee6'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:20:27.213834 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-83fb9bd2-13d2-426a-98a8-0cc5d7ca7ee6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bfceb48d37f43f299f51aa439135ac9', 'request_id': 'req-83fb9bd2-13d2-426a-98a8-0cc5d7ca7ee6'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:20:27.227167 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-83fb9bd2-13d2-426a-98a8-0cc5d7ca7ee6 admin admin] The token cache is being invalidate because project 0bfceb48d37f43f299f51aa439135ac9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:20:27.227562 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-83fb9bd2-13d2-426a-98a8-0cc5d7ca7ee6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:20:27.228274 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-83fb9bd2-13d2-426a-98a8-0cc5d7ca7ee6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:20:27.229828 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1233/2462] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:20:26 2022] DELETE /identity/v3/projects/0bfceb48d37f43f299f51aa439135ac9 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 22:20:27.239030 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f6c856-0a8a-4a98-8dfe-081584fcc900 service neutron] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:20:27.248673 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f6c856-0a8a-4a98-8dfe-081584fcc900 service neutron] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:20:27.305527 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a727b32-2a97-43d3-8daa-4f3a295e694b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:20:27.306131 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a727b32-2a97-43d3-8daa-4f3a295e694b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:27.306407 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a727b32-2a97-43d3-8daa-4f3a295e694b admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:27.306681 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a727b32-2a97-43d3-8daa-4f3a295e694b admin admin] PATH_INFO: `/v3/users/6a9a3abeb82246e6834a8022da006ba9` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:27.307794 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a727b32-2a97-43d3-8daa-4f3a295e694b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a9a3abeb82246e6834a8022da006ba9)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:20:27.309731 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a727b32-2a97-43d3-8daa-4f3a295e694b admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:20:27.521572 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-2a727b32-2a97-43d3-8daa-4f3a295e694b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a9a3abeb82246e6834a8022da006ba9'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:20:27.533434 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-2a727b32-2a97-43d3-8daa-4f3a295e694b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a9a3abeb82246e6834a8022da006ba9'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:20:27.617205 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-2a727b32-2a97-43d3-8daa-4f3a295e694b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a9a3abeb82246e6834a8022da006ba9'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:20:27.624429 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-2a727b32-2a97-43d3-8daa-4f3a295e694b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a9a3abeb82246e6834a8022da006ba9'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:20:27.625051 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-2a727b32-2a97-43d3-8daa-4f3a295e694b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a9a3abeb82246e6834a8022da006ba9'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:20:27.631144 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1230/2463] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:20:27 2022] DELETE /identity/v3/users/6a9a3abeb82246e6834a8022da006ba9 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:20:28.106787 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fb9bd2-13d2-426a-98a8-0cc5d7ca7ee6 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:20:28.109187 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-83fb9bd2-13d2-426a-98a8-0cc5d7ca7ee6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:20:28.127555 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fb9bd2-13d2-426a-98a8-0cc5d7ca7ee6 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:20:28.192680 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b901491f-8881-4b70-8da5-aa08edfaff85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:20:28.193322 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b901491f-8881-4b70-8da5-aa08edfaff85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:28.193602 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b901491f-8881-4b70-8da5-aa08edfaff85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:28.193876 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b901491f-8881-4b70-8da5-aa08edfaff85 admin admin] PATH_INFO: `/v3/projects/059f605e00d64a86b4b931ff3ce2490c` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:28.195033 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b901491f-8881-4b70-8da5-aa08edfaff85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=059f605e00d64a86b4b931ff3ce2490c)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:20:28.196996 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b901491f-8881-4b70-8da5-aa08edfaff85 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:20:28.305485 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:20:28.314388 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-b901491f-8881-4b70-8da5-aa08edfaff85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '059f605e00d64a86b4b931ff3ce2490c', 'request_id': 'req-b901491f-8881-4b70-8da5-aa08edfaff85'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:20:28.383424 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-b901491f-8881-4b70-8da5-aa08edfaff85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '059f605e00d64a86b4b931ff3ce2490c', 'request_id': 'req-b901491f-8881-4b70-8da5-aa08edfaff85'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:20:28.389646 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-b901491f-8881-4b70-8da5-aa08edfaff85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '059f605e00d64a86b4b931ff3ce2490c', 'request_id': 'req-b901491f-8881-4b70-8da5-aa08edfaff85'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:20:28.402322 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-b901491f-8881-4b70-8da5-aa08edfaff85 admin admin] The token cache is being invalidate because project 059f605e00d64a86b4b931ff3ce2490c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:20:28.402611 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-b901491f-8881-4b70-8da5-aa08edfaff85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:20:28.403323 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-b901491f-8881-4b70-8da5-aa08edfaff85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:20:28.404862 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1234/2464] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:20:28 2022] DELETE /identity/v3/projects/059f605e00d64a86b4b931ff3ce2490c => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:20:28.410404 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f2108b-bd6a-4666-9c45-4cd2ca7304d8 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:28.410930 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f2108b-bd6a-4666-9c45-4cd2ca7304d8 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:28.411207 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f2108b-bd6a-4666-9c45-4cd2ca7304d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:28.419802 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:20:28.430620 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-f7f2108b-bd6a-4666-9c45-4cd2ca7304d8 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:20:28.431831 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-f7f2108b-bd6a-4666-9c45-4cd2ca7304d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:20:28.494006 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1231/2465] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:20:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:20:28.502509 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cb0b9d-d666-4435-9d10-0f693208dcc0 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:28.503312 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cb0b9d-d666-4435-9d10-0f693208dcc0 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:28.504130 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cb0b9d-d666-4435-9d10-0f693208dcc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:28.528675 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-97cb0b9d-d666-4435-9d10-0f693208dcc0 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:20:28.529900 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-97cb0b9d-d666-4435-9d10-0f693208dcc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:20:28.540137 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1235/2466] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:20:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:20:28.544181 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f2108b-bd6a-4666-9c45-4cd2ca7304d8 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:20:28.554366 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f2108b-bd6a-4666-9c45-4cd2ca7304d8 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:20:28.559274 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebd3a07-8f55-4644-a284-fda30ad2efda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:20:28.560080 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd3a07-8f55-4644-a284-fda30ad2efda admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:28.560682 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd3a07-8f55-4644-a284-fda30ad2efda admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:28.561135 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd3a07-8f55-4644-a284-fda30ad2efda admin admin] PATH_INFO: `/v3/domains` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:28.561959 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebd3a07-8f55-4644-a284-fda30ad2efda admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 22:20:28.562566 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebd3a07-8f55-4644-a284-fda30ad2efda admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:20:28.564336 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebd3a07-8f55-4644-a284-fda30ad2efda admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:20:28.577987 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1232/2467] 10.222.0.18 () {58 vars in 1153 bytes} [Sat Aug 6 22:20:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 22:20:28.583417 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cb0b9d-d666-4435-9d10-0f693208dcc0 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:20:28.595831 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cb0b9d-d666-4435-9d10-0f693208dcc0 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:20:28.601663 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7863619-3ec7-46ac-ae90-a217e0bd3afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:20:28.602291 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7863619-3ec7-46ac-ae90-a217e0bd3afc admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:28.605691 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7863619-3ec7-46ac-ae90-a217e0bd3afc admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:28.605691 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7863619-3ec7-46ac-ae90-a217e0bd3afc admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:28.605691 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7863619-3ec7-46ac-ae90-a217e0bd3afc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:20:28.605691 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7863619-3ec7-46ac-ae90-a217e0bd3afc admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:20:28.716730 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1236/2468] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:20:28 2022] POST /identity/v3/projects => generated 379 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 22:20:28.721146 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebd3a07-8f55-4644-a284-fda30ad2efda admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:20:28.731996 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebd3a07-8f55-4644-a284-fda30ad2efda admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:20:28.792006 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee652ee-4b43-4461-8b8b-e3dbece9da7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:20:28.792648 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee652ee-4b43-4461-8b8b-e3dbece9da7a admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:28.792925 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee652ee-4b43-4461-8b8b-e3dbece9da7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:28.793258 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee652ee-4b43-4461-8b8b-e3dbece9da7a admin admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:28.793821 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee652ee-4b43-4461-8b8b-e3dbece9da7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:20:28.795733 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee652ee-4b43-4461-8b8b-e3dbece9da7a admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:20:28.966342 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1233/2469] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:20:28 2022] POST /identity/v3/users => generated 290 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:20:28.970435 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7863619-3ec7-46ac-ae90-a217e0bd3afc admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:20:28.980926 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7863619-3ec7-46ac-ae90-a217e0bd3afc admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:20:28.985524 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf67795-04b1-4a0d-9ed6-cb59d263566e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:20:28.986126 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf67795-04b1-4a0d-9ed6-cb59d263566e admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:28.986438 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf67795-04b1-4a0d-9ed6-cb59d263566e admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:28.986710 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf67795-04b1-4a0d-9ed6-cb59d263566e admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:28.987388 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf67795-04b1-4a0d-9ed6-cb59d263566e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:20:28.989156 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf67795-04b1-4a0d-9ed6-cb59d263566e admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:20:28.998959 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1237/2470] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:20:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:20:29.002805 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee652ee-4b43-4461-8b8b-e3dbece9da7a admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:20:29.013321 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee652ee-4b43-4461-8b8b-e3dbece9da7a admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:20:29.017899 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49699ea-6d7d-4bf0-8b2d-ba249f79f8d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:20:29.018487 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49699ea-6d7d-4bf0-8b2d-ba249f79f8d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:29.018781 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49699ea-6d7d-4bf0-8b2d-ba249f79f8d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:29.019700 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49699ea-6d7d-4bf0-8b2d-ba249f79f8d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:29.020349 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49699ea-6d7d-4bf0-8b2d-ba249f79f8d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:20:29.022011 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49699ea-6d7d-4bf0-8b2d-ba249f79f8d3 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:20:29.032505 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1234/2471] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:20:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:20:29.036691 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf67795-04b1-4a0d-9ed6-cb59d263566e admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:20:29.046617 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf67795-04b1-4a0d-9ed6-cb59d263566e admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:20:29.051524 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1cfc3e-aa29-4782-9022-fd763f345f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:20:29.052202 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1cfc3e-aa29-4782-9022-fd763f345f19 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:29.055175 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1cfc3e-aa29-4782-9022-fd763f345f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:29.055175 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1cfc3e-aa29-4782-9022-fd763f345f19 admin admin] PATH_INFO: `/v3/projects/5a76425203ae46a9997277eaebb7b0da/users/2df7f11d0da74ccf864f426c538ed6c1/roles/56e4a96094e9434cbbafb22579c034f5` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:29.067664 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1cfc3e-aa29-4782-9022-fd763f345f19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a76425203ae46a9997277eaebb7b0da, user_id=2df7f11d0da74ccf864f426c538ed6c1, role_id=56e4a96094e9434cbbafb22579c034f5)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:20:29.070378 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1cfc3e-aa29-4782-9022-fd763f345f19 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:20:29.172338 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1238/2472] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:20:29 2022] PUT /identity/v3/projects/5a76425203ae46a9997277eaebb7b0da/users/2df7f11d0da74ccf864f426c538ed6c1/roles/56e4a96094e9434cbbafb22579c034f5 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:20:29.184349 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c570a2-8339-4e33-bb0b-ca25781e5467 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:29.184670 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c570a2-8339-4e33-bb0b-ca25781e5467 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:29.184946 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c570a2-8339-4e33-bb0b-ca25781e5467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:29.213452 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-27c570a2-8339-4e33-bb0b-ca25781e5467 None None] MFA Rules not processed for user `2df7f11d0da74ccf864f426c538ed6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:20:29.214663 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-27c570a2-8339-4e33-bb0b-ca25781e5467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:20:29.279012 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1235/2473] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:20:29 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:20:29.283801 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1cfc3e-aa29-4782-9022-fd763f345f19 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:20:29.356973 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1cfc3e-aa29-4782-9022-fd763f345f19 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:20:29.361910 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7c62bc-3e6c-4ea5-8396-31c0d97ce691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:20:29.363311 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c62bc-3e6c-4ea5-8396-31c0d97ce691 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:29.364441 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c62bc-3e6c-4ea5-8396-31c0d97ce691 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:29.364441 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c62bc-3e6c-4ea5-8396-31c0d97ce691 admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:29.364789 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7c62bc-3e6c-4ea5-8396-31c0d97ce691 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:20:29.366886 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7c62bc-3e6c-4ea5-8396-31c0d97ce691 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:20:29.505979 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1239/2474] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:20:29 2022] POST /identity/v3/projects => generated 377 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:20:29.510143 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c570a2-8339-4e33-bb0b-ca25781e5467 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:20:29.521679 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c570a2-8339-4e33-bb0b-ca25781e5467 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:20:29.580216 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbe62a1-44dd-43b2-b6dc-fc975de37825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:20:29.580879 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe62a1-44dd-43b2-b6dc-fc975de37825 admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:29.581157 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe62a1-44dd-43b2-b6dc-fc975de37825 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:29.581429 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe62a1-44dd-43b2-b6dc-fc975de37825 admin admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:29.582086 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbe62a1-44dd-43b2-b6dc-fc975de37825 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:20:29.583992 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbe62a1-44dd-43b2-b6dc-fc975de37825 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:20:29.698086 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1236/2475] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:20:29 2022] POST /identity/v3/users => generated 295 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:20:29.703319 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7c62bc-3e6c-4ea5-8396-31c0d97ce691 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:20:29.703463 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbe62a1-44dd-43b2-b6dc-fc975de37825 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:20:29.703563 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-3d7c62bc-3e6c-4ea5-8396-31c0d97ce691 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:20:29.714924 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbe62a1-44dd-43b2-b6dc-fc975de37825 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:20:29.719970 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fc5eb7-df76-4bf9-b229-114f814b72a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:20:29.720941 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fc5eb7-df76-4bf9-b229-114f814b72a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:29.720941 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fc5eb7-df76-4bf9-b229-114f814b72a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:29.721148 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fc5eb7-df76-4bf9-b229-114f814b72a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:29.721771 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fc5eb7-df76-4bf9-b229-114f814b72a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:20:29.724429 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fc5eb7-df76-4bf9-b229-114f814b72a3 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:20:29.734672 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1237/2476] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:20:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:20:29.739272 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fc5eb7-df76-4bf9-b229-114f814b72a3 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:20:29.749084 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fc5eb7-df76-4bf9-b229-114f814b72a3 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:20:29.753962 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808f1f93-f13f-4062-8cbc-b488a5811170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:20:29.754613 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808f1f93-f13f-4062-8cbc-b488a5811170 admin admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:29.754885 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808f1f93-f13f-4062-8cbc-b488a5811170 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:29.755174 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808f1f93-f13f-4062-8cbc-b488a5811170 admin admin] PATH_INFO: `/v3/projects/556f1787a62a497e8059121037f64002/users/2b5cf849fe2b4c79bb921508daaadee7/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:29.755667 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:20:29.770101 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808f1f93-f13f-4062-8cbc-b488a5811170 admin admin] RBAC: Authorizing `identity:create_grant(project_id=556f1787a62a497e8059121037f64002, user_id=2b5cf849fe2b4c79bb921508daaadee7, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:20:29.772776 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808f1f93-f13f-4062-8cbc-b488a5811170 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:20:29.782093 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7c62bc-3e6c-4ea5-8396-31c0d97ce691 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:20:29.788381 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f819065-8dc2-4ca5-8fe3-d6bfee06b41f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a1319d33e9144859c6010843be312a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:20:29.789001 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f819065-8dc2-4ca5-8fe3-d6bfee06b41f service nova] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:29.789279 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f819065-8dc2-4ca5-8fe3-d6bfee06b41f service nova] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:29.789549 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f819065-8dc2-4ca5-8fe3-d6bfee06b41f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:29.791267 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-4f819065-8dc2-4ca5-8fe3-d6bfee06b41f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:20:29.860778 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f819065-8dc2-4ca5-8fe3-d6bfee06b41f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:20:29.862359 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f819065-8dc2-4ca5-8fe3-d6bfee06b41f service nova] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:20:29.880874 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1240/2477] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:20:29 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 22:20:29.944434 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:20:29.951427 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1238/2478] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:20:29 2022] PUT /identity/v3/projects/556f1787a62a497e8059121037f64002/users/2b5cf849fe2b4c79bb921508daaadee7/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 22:20:29.958449 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f819065-8dc2-4ca5-8fe3-d6bfee06b41f service nova] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:20:29.966626 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f819065-8dc2-4ca5-8fe3-d6bfee06b41f service nova] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:20:30.028198 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4f29bd-0136-43d4-b9a9-a03c5c6253d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:20:30.028960 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4f29bd-0136-43d4-b9a9-a03c5c6253d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:30.029246 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4f29bd-0136-43d4-b9a9-a03c5c6253d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:30.029575 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4f29bd-0136-43d4-b9a9-a03c5c6253d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:30.030168 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4f29bd-0136-43d4-b9a9-a03c5c6253d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:20:30.031786 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4f29bd-0136-43d4-b9a9-a03c5c6253d6 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:20:30.046099 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1241/2479] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:20:29 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:20:30.046809 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808f1f93-f13f-4062-8cbc-b488a5811170 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:20:30.057841 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808f1f93-f13f-4062-8cbc-b488a5811170 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:20:30.062439 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d9116d-4642-4d51-80ec-a26d36816ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:20:30.063080 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d9116d-4642-4d51-80ec-a26d36816ade admin admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:30.063375 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d9116d-4642-4d51-80ec-a26d36816ade admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:30.063632 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d9116d-4642-4d51-80ec-a26d36816ade admin admin] PATH_INFO: `/v3/projects/556f1787a62a497e8059121037f64002/users/2b5cf849fe2b4c79bb921508daaadee7/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:30.065579 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d9116d-4642-4d51-80ec-a26d36816ade admin admin] RBAC: Authorizing `identity:create_grant(project_id=556f1787a62a497e8059121037f64002, user_id=2b5cf849fe2b4c79bb921508daaadee7, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:20:30.068898 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d9116d-4642-4d51-80ec-a26d36816ade admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:20:30.079204 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1239/2480] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:20:30 2022] PUT /identity/v3/projects/556f1787a62a497e8059121037f64002/users/2b5cf849fe2b4c79bb921508daaadee7/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 22:20:30.089584 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7441213-efa9-456b-b116-e898d688b10b None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:30.089869 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7441213-efa9-456b-b116-e898d688b10b None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:30.090146 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7441213-efa9-456b-b116-e898d688b10b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:30.119075 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-f7441213-efa9-456b-b116-e898d688b10b None None] MFA Rules not processed for user `2b5cf849fe2b4c79bb921508daaadee7`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:20:30.126255 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-f7441213-efa9-456b-b116-e898d688b10b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:20:30.188342 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1242/2481] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:20:30 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:20:30.200440 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d9116d-4642-4d51-80ec-a26d36816ade admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:20:30.202141 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-e7d9116d-4642-4d51-80ec-a26d36816ade admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:20:30.260031 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d9116d-4642-4d51-80ec-a26d36816ade admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:20:30.263828 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067eb493-e248-4036-a659-4a3513f9b4a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa71c71de263478da234f3c5716a6e5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:20:30.264460 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067eb493-e248-4036-a659-4a3513f9b4a8 service cinder] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:30.269527 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067eb493-e248-4036-a659-4a3513f9b4a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:30.269527 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067eb493-e248-4036-a659-4a3513f9b4a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:30.274834 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067eb493-e248-4036-a659-4a3513f9b4a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:20:30.276469 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067eb493-e248-4036-a659-4a3513f9b4a8 service cinder] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:20:30.286609 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1240/2482] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:20:30 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:20:30.836830 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7441213-efa9-456b-b116-e898d688b10b None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:20:30.900654 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7441213-efa9-456b-b116-e898d688b10b None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:20:30.905323 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0444a14d-ba65-438f-9e27-d12b31fe78a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:20:30.905952 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0444a14d-ba65-438f-9e27-d12b31fe78a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:30.906298 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0444a14d-ba65-438f-9e27-d12b31fe78a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:30.906585 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0444a14d-ba65-438f-9e27-d12b31fe78a4 admin admin] PATH_INFO: `/v3/users/2df7f11d0da74ccf864f426c538ed6c1` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:30.907746 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0444a14d-ba65-438f-9e27-d12b31fe78a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2df7f11d0da74ccf864f426c538ed6c1)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:20:30.909879 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0444a14d-ba65-438f-9e27-d12b31fe78a4 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:20:30.993683 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:20:31.133061 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-0444a14d-ba65-438f-9e27-d12b31fe78a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2df7f11d0da74ccf864f426c538ed6c1'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:20:31.146450 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-0444a14d-ba65-438f-9e27-d12b31fe78a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2df7f11d0da74ccf864f426c538ed6c1'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:20:31.153537 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-0444a14d-ba65-438f-9e27-d12b31fe78a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2df7f11d0da74ccf864f426c538ed6c1'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:20:31.154253 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-0444a14d-ba65-438f-9e27-d12b31fe78a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2df7f11d0da74ccf864f426c538ed6c1'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:20:31.154867 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-0444a14d-ba65-438f-9e27-d12b31fe78a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2df7f11d0da74ccf864f426c538ed6c1'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:20:31.259431 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1243/2483] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:20:30 2022] DELETE /identity/v3/users/2df7f11d0da74ccf864f426c538ed6c1 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 22:20:31.263787 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba80db7-77b2-4e87-ae3f-565af03c77a3 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:31.264258 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba80db7-77b2-4e87-ae3f-565af03c77a3 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:31.264375 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba80db7-77b2-4e87-ae3f-565af03c77a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:31.283781 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-eba80db7-77b2-4e87-ae3f-565af03c77a3 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:20:31.285148 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-eba80db7-77b2-4e87-ae3f-565af03c77a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:20:31.325713 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:20:31.345279 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1241/2484] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:20:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:20:31.355572 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0444a14d-ba65-438f-9e27-d12b31fe78a4 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:20:31.358268 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-0444a14d-ba65-438f-9e27-d12b31fe78a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:20:31.415404 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0444a14d-ba65-438f-9e27-d12b31fe78a4 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:20:31.420041 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215d1b5b-93ff-432c-a697-9d54bafab234 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:20:31.420674 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215d1b5b-93ff-432c-a697-9d54bafab234 service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:31.420953 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215d1b5b-93ff-432c-a697-9d54bafab234 service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:31.421261 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215d1b5b-93ff-432c-a697-9d54bafab234 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:31.431260 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215d1b5b-93ff-432c-a697-9d54bafab234 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:20:31.432735 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215d1b5b-93ff-432c-a697-9d54bafab234 service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:20:31.443504 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1244/2485] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:20:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:20:31.780754 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba80db7-77b2-4e87-ae3f-565af03c77a3 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:20:31.782382 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-eba80db7-77b2-4e87-ae3f-565af03c77a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:20:31.858268 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba80db7-77b2-4e87-ae3f-565af03c77a3 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:20:31.864364 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfcc551-9499-4e76-97ed-943dc6544902 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a1319d33e9144859c6010843be312a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:20:31.864992 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfcc551-9499-4e76-97ed-943dc6544902 service nova] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:31.865270 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfcc551-9499-4e76-97ed-943dc6544902 service nova] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:31.865541 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfcc551-9499-4e76-97ed-943dc6544902 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:31.867381 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-0cfcc551-9499-4e76-97ed-943dc6544902 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:20:31.880906 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215d1b5b-93ff-432c-a697-9d54bafab234 service neutron] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:20:31.882268 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-215d1b5b-93ff-432c-a697-9d54bafab234 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:20:31.895092 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:20:31.899697 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215d1b5b-93ff-432c-a697-9d54bafab234 service neutron] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:20:31.906886 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a78fce-c8d9-4c44-9c68-6b2d4ae5ee61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:20:31.907750 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a78fce-c8d9-4c44-9c68-6b2d4ae5ee61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:31.907955 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a78fce-c8d9-4c44-9c68-6b2d4ae5ee61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:31.908367 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a78fce-c8d9-4c44-9c68-6b2d4ae5ee61 admin admin] PATH_INFO: `/v3/projects/5a76425203ae46a9997277eaebb7b0da` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:31.915020 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a78fce-c8d9-4c44-9c68-6b2d4ae5ee61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a76425203ae46a9997277eaebb7b0da)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:20:31.915020 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a78fce-c8d9-4c44-9c68-6b2d4ae5ee61 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:20:31.955429 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfcc551-9499-4e76-97ed-943dc6544902 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:20:31.957705 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfcc551-9499-4e76-97ed-943dc6544902 service nova] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:20:31.966790 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1242/2486] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:20:31 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:20:31.968056 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5199f97-0f10-4366-b3e1-36e9dfa8841d None None] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:31.968286 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5199f97-0f10-4366-b3e1-36e9dfa8841d None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:31.968588 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5199f97-0f10-4366-b3e1-36e9dfa8841d None None] PATH_INFO: `/` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:31.969985 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1243/2487] 10.222.0.18 () {54 vars in 921 bytes} [Sat Aug 6 22:20:31 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 22:20:31.975151 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfa1ec1-f1af-4851-b13a-620e23fde07c None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:31.975427 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfa1ec1-f1af-4851-b13a-620e23fde07c None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:31.975751 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfa1ec1-f1af-4851-b13a-620e23fde07c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:32.014684 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-2bfa1ec1-f1af-4851-b13a-620e23fde07c None None] MFA Rules not processed for user `2a1319d33e9144859c6010843be312a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:20:32.015881 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-2bfa1ec1-f1af-4851-b13a-620e23fde07c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:20:32.027023 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1244/2488] 10.222.0.18 () {58 vars in 1018 bytes} [Sat Aug 6 22:20:31 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 22:20:32.038251 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfa1ec1-f1af-4851-b13a-620e23fde07c None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:20:32.039657 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-2bfa1ec1-f1af-4851-b13a-620e23fde07c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:20:32.047135 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-25a78fce-c8d9-4c44-9c68-6b2d4ae5ee61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a76425203ae46a9997277eaebb7b0da', 'request_id': 'req-25a78fce-c8d9-4c44-9c68-6b2d4ae5ee61'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:20:32.088445 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:20:32.099620 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfa1ec1-f1af-4851-b13a-620e23fde07c None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:20:32.125403 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-25a78fce-c8d9-4c44-9c68-6b2d4ae5ee61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a76425203ae46a9997277eaebb7b0da', 'request_id': 'req-25a78fce-c8d9-4c44-9c68-6b2d4ae5ee61'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:20:32.132950 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-25a78fce-c8d9-4c44-9c68-6b2d4ae5ee61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a76425203ae46a9997277eaebb7b0da', 'request_id': 'req-25a78fce-c8d9-4c44-9c68-6b2d4ae5ee61'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:20:32.146233 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-25a78fce-c8d9-4c44-9c68-6b2d4ae5ee61 admin admin] The token cache is being invalidate because project 5a76425203ae46a9997277eaebb7b0da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:20:32.151665 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-25a78fce-c8d9-4c44-9c68-6b2d4ae5ee61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:20:32.151665 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-25a78fce-c8d9-4c44-9c68-6b2d4ae5ee61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:20:32.151665 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1245/2489] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:20:31 2022] DELETE /identity/v3/projects/5a76425203ae46a9997277eaebb7b0da => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:20:32.154283 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a78fce-c8d9-4c44-9c68-6b2d4ae5ee61 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:20:32.155743 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-25a78fce-c8d9-4c44-9c68-6b2d4ae5ee61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:20:32.157083 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26085e84-4703-44fa-ae27-ffd89eb428e6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8d38e73e105476e8de858cfa939ba20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:20:32.157738 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26085e84-4703-44fa-ae27-ffd89eb428e6 service glance] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:32.158018 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26085e84-4703-44fa-ae27-ffd89eb428e6 service glance] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:32.158283 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26085e84-4703-44fa-ae27-ffd89eb428e6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:32.160010 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-26085e84-4703-44fa-ae27-ffd89eb428e6 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:20:32.220205 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a78fce-c8d9-4c44-9c68-6b2d4ae5ee61 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:20:32.226118 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c940c9-e02e-4770-995c-38e74da66a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:20:32.226752 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c940c9-e02e-4770-995c-38e74da66a31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:32.227029 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c940c9-e02e-4770-995c-38e74da66a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:32.227300 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c940c9-e02e-4770-995c-38e74da66a31 admin admin] PATH_INFO: `/v3/users/2b5cf849fe2b4c79bb921508daaadee7` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:32.228448 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c940c9-e02e-4770-995c-38e74da66a31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b5cf849fe2b4c79bb921508daaadee7)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:20:32.230372 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c940c9-e02e-4770-995c-38e74da66a31 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:20:32.233371 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26085e84-4703-44fa-ae27-ffd89eb428e6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:20:32.234815 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26085e84-4703-44fa-ae27-ffd89eb428e6 service glance] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:20:32.246247 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1245/2490] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:20:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 22:20:32.488954 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-d9c940c9-e02e-4770-995c-38e74da66a31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b5cf849fe2b4c79bb921508daaadee7'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:20:32.502584 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-d9c940c9-e02e-4770-995c-38e74da66a31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b5cf849fe2b4c79bb921508daaadee7'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:20:32.511679 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-d9c940c9-e02e-4770-995c-38e74da66a31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b5cf849fe2b4c79bb921508daaadee7'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:20:32.512677 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-d9c940c9-e02e-4770-995c-38e74da66a31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b5cf849fe2b4c79bb921508daaadee7'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:20:32.513288 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-d9c940c9-e02e-4770-995c-38e74da66a31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b5cf849fe2b4c79bb921508daaadee7'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:20:32.606964 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1246/2491] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:20:32 2022] DELETE /identity/v3/users/2b5cf849fe2b4c79bb921508daaadee7 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 22:20:33.500578 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26085e84-4703-44fa-ae27-ffd89eb428e6 service glance] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:20:33.512326 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26085e84-4703-44fa-ae27-ffd89eb428e6 service glance] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:20:33.585656 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c58158-b62e-4c05-a42e-5d147acc6dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:20:33.586477 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c58158-b62e-4c05-a42e-5d147acc6dc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:33.586742 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c58158-b62e-4c05-a42e-5d147acc6dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:33.587017 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c58158-b62e-4c05-a42e-5d147acc6dc7 admin admin] PATH_INFO: `/v3/projects/556f1787a62a497e8059121037f64002` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:33.588652 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c58158-b62e-4c05-a42e-5d147acc6dc7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=556f1787a62a497e8059121037f64002)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:20:33.591700 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c58158-b62e-4c05-a42e-5d147acc6dc7 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:20:33.715942 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:20:33.723076 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-b7c58158-b62e-4c05-a42e-5d147acc6dc7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '556f1787a62a497e8059121037f64002', 'request_id': 'req-b7c58158-b62e-4c05-a42e-5d147acc6dc7'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:20:33.806363 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-b7c58158-b62e-4c05-a42e-5d147acc6dc7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '556f1787a62a497e8059121037f64002', 'request_id': 'req-b7c58158-b62e-4c05-a42e-5d147acc6dc7'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:20:33.812861 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-b7c58158-b62e-4c05-a42e-5d147acc6dc7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '556f1787a62a497e8059121037f64002', 'request_id': 'req-b7c58158-b62e-4c05-a42e-5d147acc6dc7'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:20:33.826323 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-b7c58158-b62e-4c05-a42e-5d147acc6dc7 admin admin] The token cache is being invalidate because project 556f1787a62a497e8059121037f64002 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:20:33.826639 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-b7c58158-b62e-4c05-a42e-5d147acc6dc7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:20:33.827343 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-b7c58158-b62e-4c05-a42e-5d147acc6dc7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:20:33.828894 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1246/2492] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:20:33 2022] DELETE /identity/v3/projects/556f1787a62a497e8059121037f64002 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:20:33.834335 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb23d0-d8a3-4fa1-9962-c97e211900c1 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:33.834836 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb23d0-d8a3-4fa1-9962-c97e211900c1 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:33.834946 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb23d0-d8a3-4fa1-9962-c97e211900c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:33.850668 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:20:33.856858 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-86cb23d0-d8a3-4fa1-9962-c97e211900c1 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:20:33.858079 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-86cb23d0-d8a3-4fa1-9962-c97e211900c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:20:33.917528 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1247/2493] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:20:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:20:33.926820 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a317b5-e61b-40b1-899c-10355eca1a90 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:33.927129 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a317b5-e61b-40b1-899c-10355eca1a90 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:33.927413 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a317b5-e61b-40b1-899c-10355eca1a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:33.951435 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-94a317b5-e61b-40b1-899c-10355eca1a90 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:20:33.952624 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-94a317b5-e61b-40b1-899c-10355eca1a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:20:33.965289 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1247/2494] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:20:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:20:33.969371 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cb23d0-d8a3-4fa1-9962-c97e211900c1 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:20:33.981898 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cb23d0-d8a3-4fa1-9962-c97e211900c1 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:20:33.987181 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cff472-0b80-4f1f-a4d0-8c1f06113b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:20:33.987898 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cff472-0b80-4f1f-a4d0-8c1f06113b0a admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:33.988151 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cff472-0b80-4f1f-a4d0-8c1f06113b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:33.988428 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cff472-0b80-4f1f-a4d0-8c1f06113b0a admin admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:33.989081 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cff472-0b80-4f1f-a4d0-8c1f06113b0a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 22:20:33.989380 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cff472-0b80-4f1f-a4d0-8c1f06113b0a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:20:33.991097 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cff472-0b80-4f1f-a4d0-8c1f06113b0a admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:20:34.003341 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1248/2495] 10.222.0.18 () {58 vars in 1153 bytes} [Sat Aug 6 22:20:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 22:20:34.007749 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a317b5-e61b-40b1-899c-10355eca1a90 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:20:34.022405 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a317b5-e61b-40b1-899c-10355eca1a90 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:20:34.027725 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7617a93d-0a6a-4640-bbdd-62352b630a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:20:34.028514 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7617a93d-0a6a-4640-bbdd-62352b630a51 admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:34.028824 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7617a93d-0a6a-4640-bbdd-62352b630a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:34.029159 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7617a93d-0a6a-4640-bbdd-62352b630a51 admin admin] PATH_INFO: `/v3/projects` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:34.029893 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7617a93d-0a6a-4640-bbdd-62352b630a51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:20:34.031890 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7617a93d-0a6a-4640-bbdd-62352b630a51 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:20:34.173899 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1248/2496] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:20:34 2022] POST /identity/v3/projects => generated 371 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:20:34.178333 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cff472-0b80-4f1f-a4d0-8c1f06113b0a admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:20:34.188982 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cff472-0b80-4f1f-a4d0-8c1f06113b0a admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:20:34.257975 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5f33dc-b972-4093-96ab-f4786805f374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:20:34.258474 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5f33dc-b972-4093-96ab-f4786805f374 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:34.258934 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5f33dc-b972-4093-96ab-f4786805f374 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:34.259201 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5f33dc-b972-4093-96ab-f4786805f374 admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:34.259828 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5f33dc-b972-4093-96ab-f4786805f374 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:20:34.261786 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5f33dc-b972-4093-96ab-f4786805f374 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:20:34.339534 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1249/2497] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:20:34 2022] POST /identity/v3/users => generated 286 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:20:34.343765 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7617a93d-0a6a-4640-bbdd-62352b630a51 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:20:34.368786 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7617a93d-0a6a-4640-bbdd-62352b630a51 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:20:34.373534 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52158b98-b254-439e-a9ce-7daca1913fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:20:34.374137 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52158b98-b254-439e-a9ce-7daca1913fdc admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:34.374412 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52158b98-b254-439e-a9ce-7daca1913fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:34.374677 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52158b98-b254-439e-a9ce-7daca1913fdc admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:34.375342 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52158b98-b254-439e-a9ce-7daca1913fdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:20:34.376968 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52158b98-b254-439e-a9ce-7daca1913fdc admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:20:34.390024 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1249/2498] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:20:34 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 22:20:34.394331 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5f33dc-b972-4093-96ab-f4786805f374 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:20:34.405154 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5f33dc-b972-4093-96ab-f4786805f374 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:20:34.410139 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476c0962-121d-474f-8c78-2fd18f954e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:20:34.410752 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c0962-121d-474f-8c78-2fd18f954e12 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:34.411031 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c0962-121d-474f-8c78-2fd18f954e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:34.411301 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c0962-121d-474f-8c78-2fd18f954e12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:34.411947 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476c0962-121d-474f-8c78-2fd18f954e12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:20:34.413600 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476c0962-121d-474f-8c78-2fd18f954e12 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:20:34.427968 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1250/2499] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:20:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:20:34.431893 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52158b98-b254-439e-a9ce-7daca1913fdc admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:20:34.444305 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52158b98-b254-439e-a9ce-7daca1913fdc admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:20:34.448803 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b414f2b2-ea98-44d4-ac47-f2a313ae2d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:20:34.449536 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b414f2b2-ea98-44d4-ac47-f2a313ae2d07 admin admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:34.449880 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b414f2b2-ea98-44d4-ac47-f2a313ae2d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:34.450277 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b414f2b2-ea98-44d4-ac47-f2a313ae2d07 admin admin] PATH_INFO: `/v3/projects/7f2abbe559614cda85e2496eaaf0afde/users/1c6163bbc2fe4e519b71588438064700/roles/56e4a96094e9434cbbafb22579c034f5` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:34.471211 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b414f2b2-ea98-44d4-ac47-f2a313ae2d07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f2abbe559614cda85e2496eaaf0afde, user_id=1c6163bbc2fe4e519b71588438064700, role_id=56e4a96094e9434cbbafb22579c034f5)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:20:34.477058 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b414f2b2-ea98-44d4-ac47-f2a313ae2d07 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:20:34.583356 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aec36b-34d0-4c50-8f8b-264c33379132 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:34.583799 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aec36b-34d0-4c50-8f8b-264c33379132 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:34.584217 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aec36b-34d0-4c50-8f8b-264c33379132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:34.587801 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1250/2500] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:20:34 2022] PUT /identity/v3/projects/7f2abbe559614cda85e2496eaaf0afde/users/1c6163bbc2fe4e519b71588438064700/roles/56e4a96094e9434cbbafb22579c034f5 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:20:34.611913 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-a9aec36b-34d0-4c50-8f8b-264c33379132 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:20:34.614167 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-a9aec36b-34d0-4c50-8f8b-264c33379132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:20:34.697475 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1251/2501] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:20:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:20:34.707715 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b414f2b2-ea98-44d4-ac47-f2a313ae2d07 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:20:34.709498 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-b414f2b2-ea98-44d4-ac47-f2a313ae2d07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:20:34.778965 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b414f2b2-ea98-44d4-ac47-f2a313ae2d07 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:20:34.783098 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ce642d-edc9-4e3e-9085-fb157f951e12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:20:34.783692 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ce642d-edc9-4e3e-9085-fb157f951e12 service neutron] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:34.783966 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ce642d-edc9-4e3e-9085-fb157f951e12 service neutron] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:34.785005 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ce642d-edc9-4e3e-9085-fb157f951e12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:34.795422 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ce642d-edc9-4e3e-9085-fb157f951e12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:20:34.796923 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ce642d-edc9-4e3e-9085-fb157f951e12 service neutron] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:20:34.808257 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1251/2502] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:20:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:20:35.573876 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fe9fe9-fd1e-4532-9644-82e2d907d17f None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:35.574406 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fe9fe9-fd1e-4532-9644-82e2d907d17f None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:35.574615 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fe9fe9-fd1e-4532-9644-82e2d907d17f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:35.596877 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-f9fe9fe9-fd1e-4532-9644-82e2d907d17f None None] MFA Rules not processed for user `d1b26ba592724dd389584bb800d877a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:20:35.598471 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-f9fe9fe9-fd1e-4532-9644-82e2d907d17f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:20:35.608873 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:20:35.661689 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1252/2503] 10.222.0.18 () {58 vars in 1009 bytes} [Sat Aug 6 22:20:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:20:35.677387 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ce642d-edc9-4e3e-9085-fb157f951e12 service neutron] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:20:35.679380 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-06ce642d-edc9-4e3e-9085-fb157f951e12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:20:35.762086 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ce642d-edc9-4e3e-9085-fb157f951e12 service neutron] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:20:35.769113 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0309d71a-8026-44bc-9644-8e30245fc233 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7fc9da7505744608dfd3bbd46a0a9d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:20:35.769113 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0309d71a-8026-44bc-9644-8e30245fc233 service swift] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:35.769418 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0309d71a-8026-44bc-9644-8e30245fc233 service swift] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:35.769647 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0309d71a-8026-44bc-9644-8e30245fc233 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:35.779272 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0309d71a-8026-44bc-9644-8e30245fc233 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:20:35.781087 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0309d71a-8026-44bc-9644-8e30245fc233 service swift] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:20:35.785664 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:20:35.791412 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1252/2504] 10.222.0.18 () {60 vars in 1384 bytes} [Sat Aug 6 22:20:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 22:20:36.204007 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3af581-bc76-46bb-9afd-ade4f4402e0f None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:36.204395 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3af581-bc76-46bb-9afd-ade4f4402e0f None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:36.204756 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3af581-bc76-46bb-9afd-ade4f4402e0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:36.241053 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-4e3af581-bc76-46bb-9afd-ade4f4402e0f None None] MFA Rules not processed for user `1c6163bbc2fe4e519b71588438064700`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:20:36.242803 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-4e3af581-bc76-46bb-9afd-ade4f4402e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:20:36.322183 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1253/2505] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:20:36 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:20:36.323246 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0309d71a-8026-44bc-9644-8e30245fc233 service swift] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:20:36.341423 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0309d71a-8026-44bc-9644-8e30245fc233 service swift] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:20:36.351185 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a14d8d9-eb4b-4642-81fb-ae6665d94d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:20:36.351911 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a14d8d9-eb4b-4642-81fb-ae6665d94d19 admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:36.352308 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a14d8d9-eb4b-4642-81fb-ae6665d94d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:36.352677 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a14d8d9-eb4b-4642-81fb-ae6665d94d19 admin admin] PATH_INFO: `/v3/projects` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:36.353469 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a14d8d9-eb4b-4642-81fb-ae6665d94d19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:20:36.355442 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a14d8d9-eb4b-4642-81fb-ae6665d94d19 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:20:36.491885 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1253/2506] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:20:36 2022] POST /identity/v3/projects => generated 371 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:20:36.493138 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3af581-bc76-46bb-9afd-ade4f4402e0f None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:20:36.508663 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3af581-bc76-46bb-9afd-ade4f4402e0f None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:20:36.600683 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4298102d-5bfb-4e02-9d2d-3583f633db23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:20:36.601307 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4298102d-5bfb-4e02-9d2d-3583f633db23 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:36.601494 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4298102d-5bfb-4e02-9d2d-3583f633db23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:36.601768 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4298102d-5bfb-4e02-9d2d-3583f633db23 admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:36.602611 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4298102d-5bfb-4e02-9d2d-3583f633db23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:20:36.604922 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4298102d-5bfb-4e02-9d2d-3583f633db23 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:20:36.692188 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1254/2507] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:20:36 2022] POST /identity/v3/users => generated 292 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 22:20:36.696061 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a14d8d9-eb4b-4642-81fb-ae6665d94d19 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:20:36.709448 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a14d8d9-eb4b-4642-81fb-ae6665d94d19 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:20:36.714820 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b26dcfd-8d5c-4809-aaed-d06c15409936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:20:36.715731 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b26dcfd-8d5c-4809-aaed-d06c15409936 admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:36.716344 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b26dcfd-8d5c-4809-aaed-d06c15409936 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:36.716876 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b26dcfd-8d5c-4809-aaed-d06c15409936 admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:36.717750 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b26dcfd-8d5c-4809-aaed-d06c15409936 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:20:36.719588 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b26dcfd-8d5c-4809-aaed-d06c15409936 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:20:36.736469 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1254/2508] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:20:36 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:20:36.737798 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4298102d-5bfb-4e02-9d2d-3583f633db23 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:20:36.748818 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4298102d-5bfb-4e02-9d2d-3583f633db23 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:20:36.753964 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4969bc52-a7b4-4133-9938-941abde1a99b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:20:36.754762 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4969bc52-a7b4-4133-9938-941abde1a99b admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:36.755148 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4969bc52-a7b4-4133-9938-941abde1a99b admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:36.755524 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4969bc52-a7b4-4133-9938-941abde1a99b admin admin] PATH_INFO: `/v3/projects/180ddf22010d49dba1d18b9d98258787/users/bf136965ec51414d885bc7ca3b6b004e/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:36.772874 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4969bc52-a7b4-4133-9938-941abde1a99b admin admin] RBAC: Authorizing `identity:create_grant(project_id=180ddf22010d49dba1d18b9d98258787, user_id=bf136965ec51414d885bc7ca3b6b004e, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:20:36.775715 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4969bc52-a7b4-4133-9938-941abde1a99b admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:20:36.838022 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1255/2509] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:20:36 2022] PUT /identity/v3/projects/180ddf22010d49dba1d18b9d98258787/users/bf136965ec51414d885bc7ca3b6b004e/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 22:20:36.839125 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b26dcfd-8d5c-4809-aaed-d06c15409936 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:20:36.850012 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b26dcfd-8d5c-4809-aaed-d06c15409936 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:20:36.917977 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb3904b-c04c-401a-bfae-62908e3f6a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:20:36.918539 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb3904b-c04c-401a-bfae-62908e3f6a7d admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:36.918816 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb3904b-c04c-401a-bfae-62908e3f6a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:36.919161 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb3904b-c04c-401a-bfae-62908e3f6a7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:36.919820 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb3904b-c04c-401a-bfae-62908e3f6a7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:20:36.921711 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb3904b-c04c-401a-bfae-62908e3f6a7d admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:20:36.937487 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4969bc52-a7b4-4133-9938-941abde1a99b admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:20:36.946992 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1255/2510] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:20:36 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:20:36.958812 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4969bc52-a7b4-4133-9938-941abde1a99b admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:20:36.964565 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2180e92-6854-42f2-bfbf-ce11d781b244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:20:36.965322 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2180e92-6854-42f2-bfbf-ce11d781b244 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:36.965601 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2180e92-6854-42f2-bfbf-ce11d781b244 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:36.965874 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2180e92-6854-42f2-bfbf-ce11d781b244 admin admin] PATH_INFO: `/v3/projects/180ddf22010d49dba1d18b9d98258787/users/bf136965ec51414d885bc7ca3b6b004e/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:36.969738 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2180e92-6854-42f2-bfbf-ce11d781b244 admin admin] RBAC: Authorizing `identity:create_grant(project_id=180ddf22010d49dba1d18b9d98258787, user_id=bf136965ec51414d885bc7ca3b6b004e, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:20:36.981391 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2180e92-6854-42f2-bfbf-ce11d781b244 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:20:37.007490 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:20:37.016502 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1256/2511] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:20:36 2022] PUT /identity/v3/projects/180ddf22010d49dba1d18b9d98258787/users/bf136965ec51414d885bc7ca3b6b004e/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 22:20:38.383076 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee900b27-7c61-4d8f-9f64-f18cd3963ab1 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:38.384125 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee900b27-7c61-4d8f-9f64-f18cd3963ab1 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:38.384593 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee900b27-7c61-4d8f-9f64-f18cd3963ab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:38.425209 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-ee900b27-7c61-4d8f-9f64-f18cd3963ab1 None None] MFA Rules not processed for user `bf136965ec51414d885bc7ca3b6b004e`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:20:38.426438 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-ee900b27-7c61-4d8f-9f64-f18cd3963ab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:20:38.519170 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1256/2512] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:20:38 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:20:38.530864 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2180e92-6854-42f2-bfbf-ce11d781b244 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:20:38.533653 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-d2180e92-6854-42f2-bfbf-ce11d781b244 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:20:38.630427 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2180e92-6854-42f2-bfbf-ce11d781b244 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:20:38.633997 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e1be01-b24f-4d1f-910b-baf789704383 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa71c71de263478da234f3c5716a6e5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:20:38.634602 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e1be01-b24f-4d1f-910b-baf789704383 service cinder] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:38.634879 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e1be01-b24f-4d1f-910b-baf789704383 service cinder] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:38.635155 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e1be01-b24f-4d1f-910b-baf789704383 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:38.695947 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e1be01-b24f-4d1f-910b-baf789704383 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:20:38.697568 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e1be01-b24f-4d1f-910b-baf789704383 service cinder] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:20:38.708345 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1257/2513] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:20:38 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:20:41.532614 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee900b27-7c61-4d8f-9f64-f18cd3963ab1 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:20:41.542959 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee900b27-7c61-4d8f-9f64-f18cd3963ab1 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:20:41.546542 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feda5bf-ed4e-4c0a-80a5-ff8b20079270 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa71c71de263478da234f3c5716a6e5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:20:41.547136 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feda5bf-ed4e-4c0a-80a5-ff8b20079270 service cinder] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:41.547415 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feda5bf-ed4e-4c0a-80a5-ff8b20079270 service cinder] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:41.547688 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feda5bf-ed4e-4c0a-80a5-ff8b20079270 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:41.557725 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feda5bf-ed4e-4c0a-80a5-ff8b20079270 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:20:41.559166 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feda5bf-ed4e-4c0a-80a5-ff8b20079270 service cinder] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:20:41.569284 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1257/2514] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:20:41 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:20:48.702741 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=48 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:20:51.565284 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=84,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:20:54.959472 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb7e46d-fca0-4e73-a945-81448a4846cc None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:54.959992 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb7e46d-fca0-4e73-a945-81448a4846cc None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:54.960094 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb7e46d-fca0-4e73-a945-81448a4846cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:55.011618 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-9fb7e46d-fca0-4e73-a945-81448a4846cc None None] MFA Rules not processed for user `d1b26ba592724dd389584bb800d877a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:20:55.012881 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-9fb7e46d-fca0-4e73-a945-81448a4846cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:20:55.079086 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1258/2515] 10.222.0.18 () {58 vars in 967 bytes} [Sat Aug 6 22:20:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:20:55.091345 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feda5bf-ed4e-4c0a-80a5-ff8b20079270 service cinder] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:20:55.167866 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feda5bf-ed4e-4c0a-80a5-ff8b20079270 service cinder] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:20:55.172804 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a66b955-1827-4ea8-bdc1-23a846ad5c56 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7fc9da7505744608dfd3bbd46a0a9d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:20:55.173393 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a66b955-1827-4ea8-bdc1-23a846ad5c56 service swift] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:20:55.174195 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a66b955-1827-4ea8-bdc1-23a846ad5c56 service swift] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:20:55.174553 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a66b955-1827-4ea8-bdc1-23a846ad5c56 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:20:55.182412 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a66b955-1827-4ea8-bdc1-23a846ad5c56 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:20:55.183966 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a66b955-1827-4ea8-bdc1-23a846ad5c56 service swift] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:20:55.192540 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1258/2516] 10.222.0.18 () {60 vars in 1384 bytes} [Sat Aug 6 22:20:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 06 22:21:01.893308 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb7e46d-fca0-4e73-a945-81448a4846cc None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:21:01.894737 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-9fb7e46d-fca0-4e73-a945-81448a4846cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:21:01.965686 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb7e46d-fca0-4e73-a945-81448a4846cc None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:21:01.970925 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a6c3e6-a706-4713-a6ab-574b26e1869b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '680d62a67ae2456f8c129e26728c3297', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:21:01.971571 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a6c3e6-a706-4713-a6ab-574b26e1869b service placement] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:21:01.971855 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a6c3e6-a706-4713-a6ab-574b26e1869b service placement] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:21:01.972187 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a6c3e6-a706-4713-a6ab-574b26e1869b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:21:01.974211 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-b8a6c3e6-a706-4713-a6ab-574b26e1869b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:21:01.985998 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a6c3e6-a706-4713-a6ab-574b26e1869b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:21:01.988244 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a6c3e6-a706-4713-a6ab-574b26e1869b service placement] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:21:01.998076 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1259/2517] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:21:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:21:02.055208 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a66b955-1827-4ea8-bdc1-23a846ad5c56 service swift] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:21:02.119244 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a66b955-1827-4ea8-bdc1-23a846ad5c56 service swift] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:21:02.124320 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd85965-44da-4d70-89eb-283b24574359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:21:02.125006 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd85965-44da-4d70-89eb-283b24574359 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:21:02.125284 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd85965-44da-4d70-89eb-283b24574359 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:21:02.125560 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd85965-44da-4d70-89eb-283b24574359 admin admin] PATH_INFO: `/v3/users/1c6163bbc2fe4e519b71588438064700` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:21:02.126711 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd85965-44da-4d70-89eb-283b24574359 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c6163bbc2fe4e519b71588438064700)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:21:02.129536 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd85965-44da-4d70-89eb-283b24574359 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:21:02.320972 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-9bd85965-44da-4d70-89eb-283b24574359 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c6163bbc2fe4e519b71588438064700'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:21:02.332786 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-9bd85965-44da-4d70-89eb-283b24574359 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c6163bbc2fe4e519b71588438064700'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:21:02.403395 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-9bd85965-44da-4d70-89eb-283b24574359 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c6163bbc2fe4e519b71588438064700'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:21:02.411281 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-9bd85965-44da-4d70-89eb-283b24574359 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c6163bbc2fe4e519b71588438064700'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:21:02.411902 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-9bd85965-44da-4d70-89eb-283b24574359 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c6163bbc2fe4e519b71588438064700'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:21:02.414035 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:21:02.418366 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1259/2518] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:21:02 2022] DELETE /identity/v3/users/1c6163bbc2fe4e519b71588438064700 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:21:02.652206 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a6c3e6-a706-4713-a6ab-574b26e1869b service placement] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:21:02.666461 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a6c3e6-a706-4713-a6ab-574b26e1869b service placement] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:21:02.723633 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d74b54a-7d5e-4d15-bcc1-0a7dea250d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:21:02.726253 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d74b54a-7d5e-4d15-bcc1-0a7dea250d00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:21:02.726253 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d74b54a-7d5e-4d15-bcc1-0a7dea250d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:21:02.726253 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d74b54a-7d5e-4d15-bcc1-0a7dea250d00 admin admin] PATH_INFO: `/v3/projects/7f2abbe559614cda85e2496eaaf0afde` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:21:02.726253 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d74b54a-7d5e-4d15-bcc1-0a7dea250d00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f2abbe559614cda85e2496eaaf0afde)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:21:02.727808 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d74b54a-7d5e-4d15-bcc1-0a7dea250d00 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:21:02.730135 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:21:02.827971 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-4d74b54a-7d5e-4d15-bcc1-0a7dea250d00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f2abbe559614cda85e2496eaaf0afde', 'request_id': 'req-4d74b54a-7d5e-4d15-bcc1-0a7dea250d00'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:21:02.906375 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-4d74b54a-7d5e-4d15-bcc1-0a7dea250d00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f2abbe559614cda85e2496eaaf0afde', 'request_id': 'req-4d74b54a-7d5e-4d15-bcc1-0a7dea250d00'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:21:02.913346 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-4d74b54a-7d5e-4d15-bcc1-0a7dea250d00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f2abbe559614cda85e2496eaaf0afde', 'request_id': 'req-4d74b54a-7d5e-4d15-bcc1-0a7dea250d00'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:21:02.924372 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-4d74b54a-7d5e-4d15-bcc1-0a7dea250d00 admin admin] The token cache is being invalidate because project 7f2abbe559614cda85e2496eaaf0afde was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:21:02.924680 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-4d74b54a-7d5e-4d15-bcc1-0a7dea250d00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:21:02.925312 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-4d74b54a-7d5e-4d15-bcc1-0a7dea250d00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:21:02.926814 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1260/2519] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:21:02 2022] DELETE /identity/v3/projects/7f2abbe559614cda85e2496eaaf0afde => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:21:02.935780 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd85965-44da-4d70-89eb-283b24574359 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:21:02.937205 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-9bd85965-44da-4d70-89eb-283b24574359 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:21:02.951340 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd85965-44da-4d70-89eb-283b24574359 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:21:03.008683 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2d36d0-4368-4302-b66e-116e6140a76f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:21:03.009312 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2d36d0-4368-4302-b66e-116e6140a76f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:21:03.009589 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2d36d0-4368-4302-b66e-116e6140a76f admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:21:03.009861 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2d36d0-4368-4302-b66e-116e6140a76f admin admin] PATH_INFO: `/v3/users/bf136965ec51414d885bc7ca3b6b004e` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:21:03.010998 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2d36d0-4368-4302-b66e-116e6140a76f admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf136965ec51414d885bc7ca3b6b004e)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:21:03.012937 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2d36d0-4368-4302-b66e-116e6140a76f admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:21:03.203203 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-7f2d36d0-4368-4302-b66e-116e6140a76f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf136965ec51414d885bc7ca3b6b004e'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:21:03.215751 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-7f2d36d0-4368-4302-b66e-116e6140a76f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf136965ec51414d885bc7ca3b6b004e'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:21:03.290781 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-7f2d36d0-4368-4302-b66e-116e6140a76f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf136965ec51414d885bc7ca3b6b004e'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:21:03.297730 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-7f2d36d0-4368-4302-b66e-116e6140a76f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf136965ec51414d885bc7ca3b6b004e'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:21:03.298350 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-7f2d36d0-4368-4302-b66e-116e6140a76f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf136965ec51414d885bc7ca3b6b004e'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:21:03.304677 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1260/2520] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:21:02 2022] DELETE /identity/v3/users/bf136965ec51414d885bc7ca3b6b004e => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:21:03.590012 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d74b54a-7d5e-4d15-bcc1-0a7dea250d00 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:21:03.598548 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-4d74b54a-7d5e-4d15-bcc1-0a7dea250d00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:21:03.607019 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d74b54a-7d5e-4d15-bcc1-0a7dea250d00 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:21:03.665457 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9709b50a-c623-418b-8dbc-ea4cf0b1018e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:21:03.666081 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9709b50a-c623-418b-8dbc-ea4cf0b1018e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:21:03.666353 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9709b50a-c623-418b-8dbc-ea4cf0b1018e admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:21:03.666620 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9709b50a-c623-418b-8dbc-ea4cf0b1018e admin admin] PATH_INFO: `/v3/projects/180ddf22010d49dba1d18b9d98258787` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:21:03.667727 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9709b50a-c623-418b-8dbc-ea4cf0b1018e admin admin] RBAC: Authorizing `identity:delete_project(project_id=180ddf22010d49dba1d18b9d98258787)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:21:03.669681 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9709b50a-c623-418b-8dbc-ea4cf0b1018e admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:21:03.851715 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-9709b50a-c623-418b-8dbc-ea4cf0b1018e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '180ddf22010d49dba1d18b9d98258787', 'request_id': 'req-9709b50a-c623-418b-8dbc-ea4cf0b1018e'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:21:03.923442 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-9709b50a-c623-418b-8dbc-ea4cf0b1018e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '180ddf22010d49dba1d18b9d98258787', 'request_id': 'req-9709b50a-c623-418b-8dbc-ea4cf0b1018e'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:21:03.930442 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-9709b50a-c623-418b-8dbc-ea4cf0b1018e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '180ddf22010d49dba1d18b9d98258787', 'request_id': 'req-9709b50a-c623-418b-8dbc-ea4cf0b1018e'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:21:03.942360 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-9709b50a-c623-418b-8dbc-ea4cf0b1018e admin admin] The token cache is being invalidate because project 180ddf22010d49dba1d18b9d98258787 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:21:03.942846 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-9709b50a-c623-418b-8dbc-ea4cf0b1018e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:21:03.943527 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-9709b50a-c623-418b-8dbc-ea4cf0b1018e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:21:03.945328 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1261/2521] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:21:03 2022] DELETE /identity/v3/projects/180ddf22010d49dba1d18b9d98258787 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:21:03.951529 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b7f273-29cf-4dd1-bada-975c9345bb7b None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:21:03.951804 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b7f273-29cf-4dd1-bada-975c9345bb7b None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:21:03.952077 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b7f273-29cf-4dd1-bada-975c9345bb7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:21:03.972217 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-e3b7f273-29cf-4dd1-bada-975c9345bb7b None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:21:03.973699 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-e3b7f273-29cf-4dd1-bada-975c9345bb7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:21:04.034738 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1261/2522] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:21:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:21:04.042464 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f5ee6-6c3b-4956-97e7-39db8dca6234 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:21:04.042749 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f5ee6-6c3b-4956-97e7-39db8dca6234 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:21:04.043018 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f5ee6-6c3b-4956-97e7-39db8dca6234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:21:04.062052 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-868f5ee6-6c3b-4956-97e7-39db8dca6234 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:21:04.063239 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-868f5ee6-6c3b-4956-97e7-39db8dca6234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:21:04.071354 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1262/2523] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:21:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:21:04.075531 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b7f273-29cf-4dd1-bada-975c9345bb7b None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:21:04.085988 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b7f273-29cf-4dd1-bada-975c9345bb7b None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:21:04.090552 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b2f06b-a43c-44d4-9c7b-40cd994440c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:21:04.091138 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b2f06b-a43c-44d4-9c7b-40cd994440c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:21:04.091413 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b2f06b-a43c-44d4-9c7b-40cd994440c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:21:04.091676 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b2f06b-a43c-44d4-9c7b-40cd994440c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:21:04.092311 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b2f06b-a43c-44d4-9c7b-40cd994440c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 22:21:04.092654 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b2f06b-a43c-44d4-9c7b-40cd994440c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:21:04.094214 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b2f06b-a43c-44d4-9c7b-40cd994440c6 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:21:04.106486 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1262/2524] 10.222.0.18 () {58 vars in 1153 bytes} [Sat Aug 6 22:21:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 22:21:04.110731 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868f5ee6-6c3b-4956-97e7-39db8dca6234 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:21:04.119926 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868f5ee6-6c3b-4956-97e7-39db8dca6234 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:21:04.124618 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17991f6-929e-4f57-b4b3-ec7e1bc655a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:21:04.125211 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17991f6-929e-4f57-b4b3-ec7e1bc655a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:21:04.125492 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17991f6-929e-4f57-b4b3-ec7e1bc655a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:21:04.125764 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17991f6-929e-4f57-b4b3-ec7e1bc655a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:21:04.126389 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17991f6-929e-4f57-b4b3-ec7e1bc655a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:21:04.128539 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17991f6-929e-4f57-b4b3-ec7e1bc655a5 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:21:04.143549 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:21:04.223745 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1263/2525] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:21:04 2022] POST /identity/v3/projects => generated 383 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:21:04.227935 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b2f06b-a43c-44d4-9c7b-40cd994440c6 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:21:04.237924 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b2f06b-a43c-44d4-9c7b-40cd994440c6 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:21:04.241479 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:21:04.294672 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a778b5-5e7c-413c-8bd6-59154e9b6b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:21:04.295305 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a778b5-5e7c-413c-8bd6-59154e9b6b24 admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:21:04.295581 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a778b5-5e7c-413c-8bd6-59154e9b6b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:21:04.295849 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a778b5-5e7c-413c-8bd6-59154e9b6b24 admin admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:21:04.296521 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a778b5-5e7c-413c-8bd6-59154e9b6b24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:21:04.298411 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a778b5-5e7c-413c-8bd6-59154e9b6b24 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:21:04.382360 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1263/2526] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:21:04 2022] POST /identity/v3/users => generated 292 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:21:04.386383 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17991f6-929e-4f57-b4b3-ec7e1bc655a5 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:21:04.397937 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17991f6-929e-4f57-b4b3-ec7e1bc655a5 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:21:04.402604 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36835166-33e5-4a25-a9da-3bd7f5ade8ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:21:04.403198 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36835166-33e5-4a25-a9da-3bd7f5ade8ac admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:21:04.403479 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36835166-33e5-4a25-a9da-3bd7f5ade8ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:21:04.403753 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36835166-33e5-4a25-a9da-3bd7f5ade8ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:21:04.404436 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36835166-33e5-4a25-a9da-3bd7f5ade8ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:21:04.406065 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36835166-33e5-4a25-a9da-3bd7f5ade8ac admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:21:04.416608 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1264/2527] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:21:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:21:04.420556 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a778b5-5e7c-413c-8bd6-59154e9b6b24 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:21:04.431000 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a778b5-5e7c-413c-8bd6-59154e9b6b24 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:21:04.435687 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcea40aa-070a-4dca-addd-aafb25cf5872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:21:04.436270 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcea40aa-070a-4dca-addd-aafb25cf5872 admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:21:04.436583 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcea40aa-070a-4dca-addd-aafb25cf5872 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:21:04.436849 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcea40aa-070a-4dca-addd-aafb25cf5872 admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:21:04.437478 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcea40aa-070a-4dca-addd-aafb25cf5872 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:21:04.439129 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcea40aa-070a-4dca-addd-aafb25cf5872 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:21:04.450562 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1264/2528] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:21:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:21:04.454552 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36835166-33e5-4a25-a9da-3bd7f5ade8ac admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:21:04.465455 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36835166-33e5-4a25-a9da-3bd7f5ade8ac admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:21:04.475114 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5d6931-c783-4570-abdc-e55e4ecbd76e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:21:04.475114 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5d6931-c783-4570-abdc-e55e4ecbd76e admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:21:04.475114 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5d6931-c783-4570-abdc-e55e4ecbd76e admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:21:04.475114 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5d6931-c783-4570-abdc-e55e4ecbd76e admin admin] PATH_INFO: `/v3/projects/e3c5c84b156a4755ad70d3b5e52441dc/users/2827a58638b94eba9f4e37f955b70b4b/roles/56e4a96094e9434cbbafb22579c034f5` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:21:04.487157 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5d6931-c783-4570-abdc-e55e4ecbd76e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3c5c84b156a4755ad70d3b5e52441dc, user_id=2827a58638b94eba9f4e37f955b70b4b, role_id=56e4a96094e9434cbbafb22579c034f5)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:21:04.489875 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5d6931-c783-4570-abdc-e55e4ecbd76e admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:21:04.574906 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1265/2529] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:21:04 2022] PUT /identity/v3/projects/e3c5c84b156a4755ad70d3b5e52441dc/users/2827a58638b94eba9f4e37f955b70b4b/roles/56e4a96094e9434cbbafb22579c034f5 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:21:04.584932 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74d5182-0b7a-4377-a09d-ed8d881eceef None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:21:04.585212 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74d5182-0b7a-4377-a09d-ed8d881eceef None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:21:04.585480 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74d5182-0b7a-4377-a09d-ed8d881eceef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:21:04.614129 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-e74d5182-0b7a-4377-a09d-ed8d881eceef None None] MFA Rules not processed for user `2827a58638b94eba9f4e37f955b70b4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:21:04.615329 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-e74d5182-0b7a-4377-a09d-ed8d881eceef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:21:04.678739 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1265/2530] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:21:04 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:21:04.683127 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5d6931-c783-4570-abdc-e55e4ecbd76e admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:21:04.743802 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5d6931-c783-4570-abdc-e55e4ecbd76e admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:21:04.748381 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d071ecc-d794-4ff6-807e-06633b03f1e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:21:04.749004 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d071ecc-d794-4ff6-807e-06633b03f1e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:21:04.749284 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d071ecc-d794-4ff6-807e-06633b03f1e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:21:04.749557 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d071ecc-d794-4ff6-807e-06633b03f1e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:21:04.750168 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d071ecc-d794-4ff6-807e-06633b03f1e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:21:04.752272 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d071ecc-d794-4ff6-807e-06633b03f1e3 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:21:04.853732 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1266/2531] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:21:04 2022] POST /identity/v3/projects => generated 385 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:21:04.857898 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74d5182-0b7a-4377-a09d-ed8d881eceef None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:21:04.868865 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74d5182-0b7a-4377-a09d-ed8d881eceef None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:21:04.925060 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d69f2b0-6e71-4925-b31b-4b432638fc10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:21:04.925661 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d69f2b0-6e71-4925-b31b-4b432638fc10 admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:21:04.925937 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d69f2b0-6e71-4925-b31b-4b432638fc10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:21:04.926209 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d69f2b0-6e71-4925-b31b-4b432638fc10 admin admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:21:04.926836 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d69f2b0-6e71-4925-b31b-4b432638fc10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:21:04.928960 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d69f2b0-6e71-4925-b31b-4b432638fc10 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:21:05.057219 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1266/2532] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:21:04 2022] POST /identity/v3/users => generated 299 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:21:05.060972 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d071ecc-d794-4ff6-807e-06633b03f1e3 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:21:05.071146 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d071ecc-d794-4ff6-807e-06633b03f1e3 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:21:05.075548 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b684f667-7a63-45dc-b6b3-75f4446e494b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:21:05.076135 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b684f667-7a63-45dc-b6b3-75f4446e494b admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:21:05.076438 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b684f667-7a63-45dc-b6b3-75f4446e494b admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:21:05.076712 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b684f667-7a63-45dc-b6b3-75f4446e494b admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:21:05.077343 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b684f667-7a63-45dc-b6b3-75f4446e494b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:21:05.078929 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b684f667-7a63-45dc-b6b3-75f4446e494b admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:21:05.089680 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1267/2533] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:21:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:21:05.093766 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d69f2b0-6e71-4925-b31b-4b432638fc10 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:21:05.101177 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:21:05.104952 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d69f2b0-6e71-4925-b31b-4b432638fc10 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:21:05.109625 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8fbabf-2fbb-4dc8-ab41-563c27bbba86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:21:05.110294 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8fbabf-2fbb-4dc8-ab41-563c27bbba86 admin admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:21:05.112917 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8fbabf-2fbb-4dc8-ab41-563c27bbba86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:21:05.112917 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8fbabf-2fbb-4dc8-ab41-563c27bbba86 admin admin] PATH_INFO: `/v3/projects/2f1c698559454a95b36b3e79025e0f84/users/6dc5c81b95ed433bbe7785605bb600c9/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:21:05.125674 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8fbabf-2fbb-4dc8-ab41-563c27bbba86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f1c698559454a95b36b3e79025e0f84, user_id=6dc5c81b95ed433bbe7785605bb600c9, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:21:05.128531 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8fbabf-2fbb-4dc8-ab41-563c27bbba86 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:21:05.239165 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1267/2534] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:21:05 2022] PUT /identity/v3/projects/2f1c698559454a95b36b3e79025e0f84/users/6dc5c81b95ed433bbe7785605bb600c9/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:21:05.243231 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b684f667-7a63-45dc-b6b3-75f4446e494b admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:21:05.255949 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b684f667-7a63-45dc-b6b3-75f4446e494b admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:21:05.310353 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf7a8a8-6901-48ee-b670-5686c6af906e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:21:05.310944 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf7a8a8-6901-48ee-b670-5686c6af906e admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:21:05.311247 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf7a8a8-6901-48ee-b670-5686c6af906e admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:21:05.311515 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf7a8a8-6901-48ee-b670-5686c6af906e admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:21:05.312140 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf7a8a8-6901-48ee-b670-5686c6af906e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:21:05.313782 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf7a8a8-6901-48ee-b670-5686c6af906e admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:21:05.323638 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1268/2535] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:21:05 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 22:21:05.327888 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8fbabf-2fbb-4dc8-ab41-563c27bbba86 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:21:05.338738 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8fbabf-2fbb-4dc8-ab41-563c27bbba86 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:21:05.343361 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7259166-bee8-4614-889d-059cce0ead16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:21:05.344034 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7259166-bee8-4614-889d-059cce0ead16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:21:05.344307 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7259166-bee8-4614-889d-059cce0ead16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:21:05.344604 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7259166-bee8-4614-889d-059cce0ead16 admin admin] PATH_INFO: `/v3/projects/2f1c698559454a95b36b3e79025e0f84/users/6dc5c81b95ed433bbe7785605bb600c9/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:21:05.346911 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7259166-bee8-4614-889d-059cce0ead16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f1c698559454a95b36b3e79025e0f84, user_id=6dc5c81b95ed433bbe7785605bb600c9, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:21:05.349849 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7259166-bee8-4614-889d-059cce0ead16 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:21:05.360841 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1268/2536] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:21:05 2022] PUT /identity/v3/projects/2f1c698559454a95b36b3e79025e0f84/users/6dc5c81b95ed433bbe7785605bb600c9/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:21:05.372285 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab82bd03-04ba-478a-b493-64da2cb761e7 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:21:05.372604 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab82bd03-04ba-478a-b493-64da2cb761e7 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:21:05.372875 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab82bd03-04ba-478a-b493-64da2cb761e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:21:05.401079 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-ab82bd03-04ba-478a-b493-64da2cb761e7 None None] MFA Rules not processed for user `6dc5c81b95ed433bbe7785605bb600c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:21:05.402283 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-ab82bd03-04ba-478a-b493-64da2cb761e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:21:05.418622 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:21:05.470443 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1269/2537] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:21:05 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:21:05.481571 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7259166-bee8-4614-889d-059cce0ead16 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:21:05.483225 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-c7259166-bee8-4614-889d-059cce0ead16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:21:05.541221 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7259166-bee8-4614-889d-059cce0ead16 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:21:05.544924 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f554fd-cfa9-40bd-90e0-289a3d9529cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa71c71de263478da234f3c5716a6e5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:21:05.545525 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f554fd-cfa9-40bd-90e0-289a3d9529cd service cinder] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:21:05.545802 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f554fd-cfa9-40bd-90e0-289a3d9529cd service cinder] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:21:05.546073 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f554fd-cfa9-40bd-90e0-289a3d9529cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:21:05.601489 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f554fd-cfa9-40bd-90e0-289a3d9529cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:21:05.602910 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f554fd-cfa9-40bd-90e0-289a3d9529cd service cinder] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:21:05.612307 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1269/2538] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:21:05 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:21:11.957887 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab82bd03-04ba-478a-b493-64da2cb761e7 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:21:11.967695 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab82bd03-04ba-478a-b493-64da2cb761e7 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:21:11.971870 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d172cde-7ad8-4f85-ae6c-3ae2d8fb0001 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa71c71de263478da234f3c5716a6e5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:21:11.972744 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d172cde-7ad8-4f85-ae6c-3ae2d8fb0001 service cinder] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:21:11.973020 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d172cde-7ad8-4f85-ae6c-3ae2d8fb0001 service cinder] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:21:11.973289 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d172cde-7ad8-4f85-ae6c-3ae2d8fb0001 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:21:11.982529 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d172cde-7ad8-4f85-ae6c-3ae2d8fb0001 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:21:11.983933 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d172cde-7ad8-4f85-ae6c-3ae2d8fb0001 service cinder] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:21:11.993367 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1270/2539] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:21:11 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:21:15.607694 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=61,keystone:INSERT=2 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:21:18.045285 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f554fd-cfa9-40bd-90e0-289a3d9529cd service cinder] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:21:18.106621 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f554fd-cfa9-40bd-90e0-289a3d9529cd service cinder] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:21:18.111149 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9f888f-fc0f-4ac5-9e09-7852b4971315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:21:18.111766 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9f888f-fc0f-4ac5-9e09-7852b4971315 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:21:18.112042 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9f888f-fc0f-4ac5-9e09-7852b4971315 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:21:18.112318 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9f888f-fc0f-4ac5-9e09-7852b4971315 admin admin] PATH_INFO: `/v3/users/2827a58638b94eba9f4e37f955b70b4b` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:21:18.113463 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9f888f-fc0f-4ac5-9e09-7852b4971315 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2827a58638b94eba9f4e37f955b70b4b)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:21:18.115370 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9f888f-fc0f-4ac5-9e09-7852b4971315 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:21:18.330141 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-9d9f888f-fc0f-4ac5-9e09-7852b4971315 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2827a58638b94eba9f4e37f955b70b4b'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:21:18.341867 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-9d9f888f-fc0f-4ac5-9e09-7852b4971315 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2827a58638b94eba9f4e37f955b70b4b'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:21:18.416409 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-9d9f888f-fc0f-4ac5-9e09-7852b4971315 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2827a58638b94eba9f4e37f955b70b4b'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:21:18.423884 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-9d9f888f-fc0f-4ac5-9e09-7852b4971315 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2827a58638b94eba9f4e37f955b70b4b'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:21:18.424549 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-9d9f888f-fc0f-4ac5-9e09-7852b4971315 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2827a58638b94eba9f4e37f955b70b4b'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:21:18.431447 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1270/2540] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:21:18 2022] DELETE /identity/v3/users/2827a58638b94eba9f4e37f955b70b4b => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:21:18.436242 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae1ab1-a52a-448f-9009-3f31c29cac15 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:21:18.436731 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae1ab1-a52a-448f-9009-3f31c29cac15 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:21:18.436844 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae1ab1-a52a-448f-9009-3f31c29cac15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:21:18.455704 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-ffae1ab1-a52a-448f-9009-3f31c29cac15 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:21:18.456940 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-ffae1ab1-a52a-448f-9009-3f31c29cac15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:21:18.514372 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1271/2541] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:21:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:21:18.524668 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9f888f-fc0f-4ac5-9e09-7852b4971315 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:21:18.526055 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-9d9f888f-fc0f-4ac5-9e09-7852b4971315 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:21:18.584330 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9f888f-fc0f-4ac5-9e09-7852b4971315 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:21:18.588682 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f184b53-0e74-4667-92b2-f0320bc330e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:21:18.589280 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f184b53-0e74-4667-92b2-f0320bc330e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:21:18.589553 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f184b53-0e74-4667-92b2-f0320bc330e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:21:18.589852 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f184b53-0e74-4667-92b2-f0320bc330e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:21:18.598899 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f184b53-0e74-4667-92b2-f0320bc330e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:21:18.600303 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f184b53-0e74-4667-92b2-f0320bc330e5 service neutron] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:21:18.610261 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1271/2542] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:21:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:21:19.006127 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffae1ab1-a52a-448f-9009-3f31c29cac15 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:21:19.007564 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-ffae1ab1-a52a-448f-9009-3f31c29cac15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:21:19.021184 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffae1ab1-a52a-448f-9009-3f31c29cac15 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:21:19.026770 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ea1a9f-d7dc-419e-a7d6-2d71540d87a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:21:19.027399 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ea1a9f-d7dc-419e-a7d6-2d71540d87a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:21:19.027675 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ea1a9f-d7dc-419e-a7d6-2d71540d87a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:21:19.027941 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ea1a9f-d7dc-419e-a7d6-2d71540d87a8 admin admin] PATH_INFO: `/v3/projects/e3c5c84b156a4755ad70d3b5e52441dc` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:21:19.030113 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ea1a9f-d7dc-419e-a7d6-2d71540d87a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3c5c84b156a4755ad70d3b5e52441dc)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:21:19.032208 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ea1a9f-d7dc-419e-a7d6-2d71540d87a8 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:21:19.124630 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-a2ea1a9f-d7dc-419e-a7d6-2d71540d87a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3c5c84b156a4755ad70d3b5e52441dc', 'request_id': 'req-a2ea1a9f-d7dc-419e-a7d6-2d71540d87a8'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:21:19.198664 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-a2ea1a9f-d7dc-419e-a7d6-2d71540d87a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3c5c84b156a4755ad70d3b5e52441dc', 'request_id': 'req-a2ea1a9f-d7dc-419e-a7d6-2d71540d87a8'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:21:19.204905 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-a2ea1a9f-d7dc-419e-a7d6-2d71540d87a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3c5c84b156a4755ad70d3b5e52441dc', 'request_id': 'req-a2ea1a9f-d7dc-419e-a7d6-2d71540d87a8'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:21:19.216534 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-a2ea1a9f-d7dc-419e-a7d6-2d71540d87a8 admin admin] The token cache is being invalidate because project e3c5c84b156a4755ad70d3b5e52441dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:21:19.216820 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-a2ea1a9f-d7dc-419e-a7d6-2d71540d87a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:21:19.217449 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-a2ea1a9f-d7dc-419e-a7d6-2d71540d87a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:21:19.218967 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1272/2543] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:21:19 2022] DELETE /identity/v3/projects/e3c5c84b156a4755ad70d3b5e52441dc => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:21:19.223275 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f184b53-0e74-4667-92b2-f0320bc330e5 service neutron] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:21:19.233068 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f184b53-0e74-4667-92b2-f0320bc330e5 service neutron] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:21:19.251035 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:21:19.289740 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedc8e82-aa7b-410b-aae0-8f731273a4fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:21:19.290600 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedc8e82-aa7b-410b-aae0-8f731273a4fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:21:19.291056 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedc8e82-aa7b-410b-aae0-8f731273a4fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:21:19.291774 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedc8e82-aa7b-410b-aae0-8f731273a4fa admin admin] PATH_INFO: `/v3/users/6dc5c81b95ed433bbe7785605bb600c9` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:21:19.292997 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedc8e82-aa7b-410b-aae0-8f731273a4fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=6dc5c81b95ed433bbe7785605bb600c9)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:21:19.294911 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedc8e82-aa7b-410b-aae0-8f731273a4fa admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:21:19.446535 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-cedc8e82-aa7b-410b-aae0-8f731273a4fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6dc5c81b95ed433bbe7785605bb600c9'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:21:19.458220 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-cedc8e82-aa7b-410b-aae0-8f731273a4fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6dc5c81b95ed433bbe7785605bb600c9'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:21:19.547117 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-cedc8e82-aa7b-410b-aae0-8f731273a4fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6dc5c81b95ed433bbe7785605bb600c9'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:21:19.555450 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-cedc8e82-aa7b-410b-aae0-8f731273a4fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6dc5c81b95ed433bbe7785605bb600c9'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:21:19.556855 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-cedc8e82-aa7b-410b-aae0-8f731273a4fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6dc5c81b95ed433bbe7785605bb600c9'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:21:19.562166 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1272/2544] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:21:19 2022] DELETE /identity/v3/users/6dc5c81b95ed433bbe7785605bb600c9 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:21:20.209567 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ea1a9f-d7dc-419e-a7d6-2d71540d87a8 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:21:20.211048 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-a2ea1a9f-d7dc-419e-a7d6-2d71540d87a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:21:20.224425 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ea1a9f-d7dc-419e-a7d6-2d71540d87a8 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:21:20.253303 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:21:20.289703 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749ab047-019a-48c4-9fd8-f72bf849f6f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:21:20.290329 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ab047-019a-48c4-9fd8-f72bf849f6f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:21:20.290609 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ab047-019a-48c4-9fd8-f72bf849f6f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:21:20.290879 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ab047-019a-48c4-9fd8-f72bf849f6f9 admin admin] PATH_INFO: `/v3/projects/2f1c698559454a95b36b3e79025e0f84` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:21:20.291984 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749ab047-019a-48c4-9fd8-f72bf849f6f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f1c698559454a95b36b3e79025e0f84)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:21:20.293920 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749ab047-019a-48c4-9fd8-f72bf849f6f9 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:21:20.410822 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-749ab047-019a-48c4-9fd8-f72bf849f6f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f1c698559454a95b36b3e79025e0f84', 'request_id': 'req-749ab047-019a-48c4-9fd8-f72bf849f6f9'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:21:20.507266 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-749ab047-019a-48c4-9fd8-f72bf849f6f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f1c698559454a95b36b3e79025e0f84', 'request_id': 'req-749ab047-019a-48c4-9fd8-f72bf849f6f9'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:21:20.513297 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-749ab047-019a-48c4-9fd8-f72bf849f6f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f1c698559454a95b36b3e79025e0f84', 'request_id': 'req-749ab047-019a-48c4-9fd8-f72bf849f6f9'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:21:20.524276 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-749ab047-019a-48c4-9fd8-f72bf849f6f9 admin admin] The token cache is being invalidate because project 2f1c698559454a95b36b3e79025e0f84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:21:20.524600 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-749ab047-019a-48c4-9fd8-f72bf849f6f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:21:20.525229 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-749ab047-019a-48c4-9fd8-f72bf849f6f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:21:20.526699 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1273/2545] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:21:20 2022] DELETE /identity/v3/projects/2f1c698559454a95b36b3e79025e0f84 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:21:20.532553 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2fa9d0-0822-491b-92a6-a026ff35d112 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:21:20.533048 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2fa9d0-0822-491b-92a6-a026ff35d112 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:21:20.533169 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2fa9d0-0822-491b-92a6-a026ff35d112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:21:20.552797 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-0d2fa9d0-0822-491b-92a6-a026ff35d112 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:21:20.553981 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-0d2fa9d0-0822-491b-92a6-a026ff35d112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:21:20.613584 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1273/2546] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:21:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:21:20.622230 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6cf66b-0d59-4aa5-ad31-e0279296ee84 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:21:20.622511 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6cf66b-0d59-4aa5-ad31-e0279296ee84 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:21:20.622783 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6cf66b-0d59-4aa5-ad31-e0279296ee84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:21:20.642123 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-1d6cf66b-0d59-4aa5-ad31-e0279296ee84 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:21:20.643309 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-1d6cf66b-0d59-4aa5-ad31-e0279296ee84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:21:20.651145 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1274/2547] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:21:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:21:20.655437 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2fa9d0-0822-491b-92a6-a026ff35d112 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:21:20.665622 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2fa9d0-0822-491b-92a6-a026ff35d112 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:21:20.670249 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc74f91-c98b-4d58-a03b-c1318c37c97a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:21:20.670844 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc74f91-c98b-4d58-a03b-c1318c37c97a admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:21:20.671122 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc74f91-c98b-4d58-a03b-c1318c37c97a admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:21:20.671394 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc74f91-c98b-4d58-a03b-c1318c37c97a admin admin] PATH_INFO: `/v3/domains` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:21:20.672074 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc74f91-c98b-4d58-a03b-c1318c37c97a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 22:21:20.672372 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc74f91-c98b-4d58-a03b-c1318c37c97a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:21:20.674003 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc74f91-c98b-4d58-a03b-c1318c37c97a admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:21:20.686831 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1274/2548] 10.222.0.18 () {58 vars in 1153 bytes} [Sat Aug 6 22:21:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 22:21:20.691410 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6cf66b-0d59-4aa5-ad31-e0279296ee84 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:21:20.700616 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6cf66b-0d59-4aa5-ad31-e0279296ee84 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:21:20.705204 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a4566b-56a0-4e0b-a7a1-403dea9e043c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:21:20.705787 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a4566b-56a0-4e0b-a7a1-403dea9e043c admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:21:20.706061 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a4566b-56a0-4e0b-a7a1-403dea9e043c admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:21:20.706328 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a4566b-56a0-4e0b-a7a1-403dea9e043c admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:21:20.706936 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a4566b-56a0-4e0b-a7a1-403dea9e043c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:21:20.708927 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a4566b-56a0-4e0b-a7a1-403dea9e043c admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:21:20.806670 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1275/2549] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:21:20 2022] POST /identity/v3/projects => generated 361 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:21:20.811088 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc74f91-c98b-4d58-a03b-c1318c37c97a admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:21:20.821508 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc74f91-c98b-4d58-a03b-c1318c37c97a admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:21:20.848218 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:21:20.879116 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81d7c6c-c967-4311-9365-0584d3193ebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:21:20.879715 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81d7c6c-c967-4311-9365-0584d3193ebb admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:21:20.879988 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81d7c6c-c967-4311-9365-0584d3193ebb admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:21:20.880259 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81d7c6c-c967-4311-9365-0584d3193ebb admin admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:21:20.880916 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81d7c6c-c967-4311-9365-0584d3193ebb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:21:20.882821 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81d7c6c-c967-4311-9365-0584d3193ebb admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:21:20.970085 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1275/2550] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:21:20 2022] POST /identity/v3/users => generated 281 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 22:21:20.974663 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a4566b-56a0-4e0b-a7a1-403dea9e043c admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:21:20.986267 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a4566b-56a0-4e0b-a7a1-403dea9e043c admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:21:20.991004 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16424e2-aee7-413b-aa77-cbfc2a64fd4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:21:20.991597 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16424e2-aee7-413b-aa77-cbfc2a64fd4d admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:21:20.991879 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16424e2-aee7-413b-aa77-cbfc2a64fd4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:21:20.992154 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16424e2-aee7-413b-aa77-cbfc2a64fd4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:21:20.992850 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16424e2-aee7-413b-aa77-cbfc2a64fd4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:21:20.994551 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16424e2-aee7-413b-aa77-cbfc2a64fd4d admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:21:21.004792 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1276/2551] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:21:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:21:21.008722 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81d7c6c-c967-4311-9365-0584d3193ebb admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:21:21.018765 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81d7c6c-c967-4311-9365-0584d3193ebb admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:21:21.023406 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32b79c2-88d3-4e5b-9bc8-ca1f4d24bac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:21:21.023998 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b79c2-88d3-4e5b-9bc8-ca1f4d24bac8 admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:21:21.024277 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b79c2-88d3-4e5b-9bc8-ca1f4d24bac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:21:21.025079 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b79c2-88d3-4e5b-9bc8-ca1f4d24bac8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:21:21.025714 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32b79c2-88d3-4e5b-9bc8-ca1f4d24bac8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:21:21.027280 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32b79c2-88d3-4e5b-9bc8-ca1f4d24bac8 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:21:21.037040 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1276/2552] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:21:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:21:21.041628 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16424e2-aee7-413b-aa77-cbfc2a64fd4d admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:21:21.051491 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16424e2-aee7-413b-aa77-cbfc2a64fd4d admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:21:21.056088 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175fde50-6d43-4813-8f4c-1de3c8b7f07f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:21:21.056821 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175fde50-6d43-4813-8f4c-1de3c8b7f07f admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:21:21.057048 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175fde50-6d43-4813-8f4c-1de3c8b7f07f admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:21:21.057326 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175fde50-6d43-4813-8f4c-1de3c8b7f07f admin admin] PATH_INFO: `/v3/projects/89a0eb5d187a477d87771659142a87b1/users/c3932a56dce049189c8c684ed6223e71/roles/56e4a96094e9434cbbafb22579c034f5` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:21:21.071546 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175fde50-6d43-4813-8f4c-1de3c8b7f07f admin admin] RBAC: Authorizing `identity:create_grant(project_id=89a0eb5d187a477d87771659142a87b1, user_id=c3932a56dce049189c8c684ed6223e71, role_id=56e4a96094e9434cbbafb22579c034f5)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:21:21.074243 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175fde50-6d43-4813-8f4c-1de3c8b7f07f admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:21:21.141669 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1277/2553] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:21:21 2022] PUT /identity/v3/projects/89a0eb5d187a477d87771659142a87b1/users/c3932a56dce049189c8c684ed6223e71/roles/56e4a96094e9434cbbafb22579c034f5 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:21:21.153642 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136f972f-5ef0-40fa-9eb3-c0d3dfaa6766 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:21:21.153923 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136f972f-5ef0-40fa-9eb3-c0d3dfaa6766 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:21:21.154189 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136f972f-5ef0-40fa-9eb3-c0d3dfaa6766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:21:21.181326 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-136f972f-5ef0-40fa-9eb3-c0d3dfaa6766 None None] MFA Rules not processed for user `c3932a56dce049189c8c684ed6223e71`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:21:21.182595 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-136f972f-5ef0-40fa-9eb3-c0d3dfaa6766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:21:21.245378 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1277/2554] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:21:21 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:21:21.281306 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175fde50-6d43-4813-8f4c-1de3c8b7f07f admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:21:21.283073 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-175fde50-6d43-4813-8f4c-1de3c8b7f07f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:21:21.340255 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175fde50-6d43-4813-8f4c-1de3c8b7f07f admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:21:21.344787 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99793cfd-3a85-4dd9-967d-411211b267ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa71c71de263478da234f3c5716a6e5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:21:21.345379 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99793cfd-3a85-4dd9-967d-411211b267ea service cinder] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:21:21.345656 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99793cfd-3a85-4dd9-967d-411211b267ea service cinder] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:21:21.345924 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99793cfd-3a85-4dd9-967d-411211b267ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:21:21.351693 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:21:21.355711 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99793cfd-3a85-4dd9-967d-411211b267ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:21:21.357182 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99793cfd-3a85-4dd9-967d-411211b267ea service cinder] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:21:21.366460 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1278/2555] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:21:21 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:21:21.382260 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136f972f-5ef0-40fa-9eb3-c0d3dfaa6766 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:21:21.443390 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136f972f-5ef0-40fa-9eb3-c0d3dfaa6766 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:21:21.448625 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef03029-b3a8-4a4a-9c58-515f98373949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:21:21.449243 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef03029-b3a8-4a4a-9c58-515f98373949 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:21:21.449523 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef03029-b3a8-4a4a-9c58-515f98373949 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:21:21.449794 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef03029-b3a8-4a4a-9c58-515f98373949 admin admin] PATH_INFO: `/v3/users/c3932a56dce049189c8c684ed6223e71` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:21:21.450922 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef03029-b3a8-4a4a-9c58-515f98373949 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3932a56dce049189c8c684ed6223e71)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:21:21.452865 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef03029-b3a8-4a4a-9c58-515f98373949 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:21:21.662241 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:21:21.668117 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-0ef03029-b3a8-4a4a-9c58-515f98373949 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3932a56dce049189c8c684ed6223e71'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:21:21.679816 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-0ef03029-b3a8-4a4a-9c58-515f98373949 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3932a56dce049189c8c684ed6223e71'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:21:21.785698 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-0ef03029-b3a8-4a4a-9c58-515f98373949 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3932a56dce049189c8c684ed6223e71'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:21:21.793086 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-0ef03029-b3a8-4a4a-9c58-515f98373949 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3932a56dce049189c8c684ed6223e71'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:21:21.793755 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-0ef03029-b3a8-4a4a-9c58-515f98373949 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3932a56dce049189c8c684ed6223e71'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:21:21.799701 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1278/2556] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:21:21 2022] DELETE /identity/v3/users/c3932a56dce049189c8c684ed6223e71 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:21:21.804173 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f7bf8-c6fb-42cc-9ea7-1aa7366523e2 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:21:21.804509 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f7bf8-c6fb-42cc-9ea7-1aa7366523e2 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:21:21.804805 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f7bf8-c6fb-42cc-9ea7-1aa7366523e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:21:21.825394 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-b33f7bf8-c6fb-42cc-9ea7-1aa7366523e2 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:21:21.826607 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-b33f7bf8-c6fb-42cc-9ea7-1aa7366523e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:21:21.885749 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1279/2557] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:21:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:21:21.895364 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef03029-b3a8-4a4a-9c58-515f98373949 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:21:21.896808 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-0ef03029-b3a8-4a4a-9c58-515f98373949 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:21:21.955026 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef03029-b3a8-4a4a-9c58-515f98373949 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:21:21.958536 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b40794-ca89-4392-a618-47a211781458 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:21:21.959140 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b40794-ca89-4392-a618-47a211781458 service neutron] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:21:21.959425 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b40794-ca89-4392-a618-47a211781458 service neutron] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:21:21.959699 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b40794-ca89-4392-a618-47a211781458 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:21:21.968819 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b40794-ca89-4392-a618-47a211781458 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:21:21.970249 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b40794-ca89-4392-a618-47a211781458 service neutron] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:21:21.980569 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1279/2558] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:21:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:21:22.384606 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33f7bf8-c6fb-42cc-9ea7-1aa7366523e2 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:21:22.386144 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-b33f7bf8-c6fb-42cc-9ea7-1aa7366523e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:21:22.399086 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33f7bf8-c6fb-42cc-9ea7-1aa7366523e2 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:21:22.404066 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7daf2c-36e8-468d-b6c4-c71632d786d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:21:22.404789 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7daf2c-36e8-468d-b6c4-c71632d786d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:21:22.405065 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7daf2c-36e8-468d-b6c4-c71632d786d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:21:22.405336 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7daf2c-36e8-468d-b6c4-c71632d786d8 admin admin] PATH_INFO: `/v3/projects/89a0eb5d187a477d87771659142a87b1` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:21:22.406479 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7daf2c-36e8-468d-b6c4-c71632d786d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89a0eb5d187a477d87771659142a87b1)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:21:22.408444 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7daf2c-36e8-468d-b6c4-c71632d786d8 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:21:22.512817 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-4f7daf2c-36e8-468d-b6c4-c71632d786d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89a0eb5d187a477d87771659142a87b1', 'request_id': 'req-4f7daf2c-36e8-468d-b6c4-c71632d786d8'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:21:22.603258 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-4f7daf2c-36e8-468d-b6c4-c71632d786d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89a0eb5d187a477d87771659142a87b1', 'request_id': 'req-4f7daf2c-36e8-468d-b6c4-c71632d786d8'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:21:22.609395 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-4f7daf2c-36e8-468d-b6c4-c71632d786d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89a0eb5d187a477d87771659142a87b1', 'request_id': 'req-4f7daf2c-36e8-468d-b6c4-c71632d786d8'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:21:22.621070 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-4f7daf2c-36e8-468d-b6c4-c71632d786d8 admin admin] The token cache is being invalidate because project 89a0eb5d187a477d87771659142a87b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:21:22.621356 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-4f7daf2c-36e8-468d-b6c4-c71632d786d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:21:22.621985 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-4f7daf2c-36e8-468d-b6c4-c71632d786d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:21:22.623463 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1280/2559] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:21:22 2022] DELETE /identity/v3/projects/89a0eb5d187a477d87771659142a87b1 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:21:22.628681 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36462ae5-ee2f-4f4e-8da2-c953323026da None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:21:22.628967 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36462ae5-ee2f-4f4e-8da2-c953323026da None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:21:22.629241 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36462ae5-ee2f-4f4e-8da2-c953323026da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:21:22.649754 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-36462ae5-ee2f-4f4e-8da2-c953323026da None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:21:22.650943 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-36462ae5-ee2f-4f4e-8da2-c953323026da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:21:22.709846 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1280/2560] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:21:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:21:22.718321 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62c64d5-4bd7-4593-be9b-3922dab7f749 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:21:22.718724 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62c64d5-4bd7-4593-be9b-3922dab7f749 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:21:22.718985 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62c64d5-4bd7-4593-be9b-3922dab7f749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:21:22.738871 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-d62c64d5-4bd7-4593-be9b-3922dab7f749 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:21:22.740076 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-d62c64d5-4bd7-4593-be9b-3922dab7f749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:21:22.748192 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1281/2561] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:21:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:21:22.752341 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36462ae5-ee2f-4f4e-8da2-c953323026da None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:21:22.762584 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36462ae5-ee2f-4f4e-8da2-c953323026da None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:21:22.767240 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ddfa4f-a9f2-4239-87bd-b68d84f01b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:21:22.767784 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ddfa4f-a9f2-4239-87bd-b68d84f01b26 admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:21:22.768044 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ddfa4f-a9f2-4239-87bd-b68d84f01b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:21:22.768310 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ddfa4f-a9f2-4239-87bd-b68d84f01b26 admin admin] PATH_INFO: `/v3/domains` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:21:22.768980 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ddfa4f-a9f2-4239-87bd-b68d84f01b26 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 22:21:22.769277 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ddfa4f-a9f2-4239-87bd-b68d84f01b26 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:21:22.770848 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ddfa4f-a9f2-4239-87bd-b68d84f01b26 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:21:22.782942 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1281/2562] 10.222.0.18 () {58 vars in 1153 bytes} [Sat Aug 6 22:21:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 22:21:22.787243 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62c64d5-4bd7-4593-be9b-3922dab7f749 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:21:22.796687 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62c64d5-4bd7-4593-be9b-3922dab7f749 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:21:22.801340 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5acb1d-f5b0-40c2-9764-996a7c9bb40a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:21:22.801930 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5acb1d-f5b0-40c2-9764-996a7c9bb40a admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:21:22.802205 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5acb1d-f5b0-40c2-9764-996a7c9bb40a admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:21:22.802472 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5acb1d-f5b0-40c2-9764-996a7c9bb40a admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:21:22.803107 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5acb1d-f5b0-40c2-9764-996a7c9bb40a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:21:22.805041 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5acb1d-f5b0-40c2-9764-996a7c9bb40a admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:21:22.896987 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1282/2563] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:21:22 2022] POST /identity/v3/projects => generated 377 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:21:22.901794 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ddfa4f-a9f2-4239-87bd-b68d84f01b26 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:21:22.912029 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ddfa4f-a9f2-4239-87bd-b68d84f01b26 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:21:22.955003 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:21:22.968252 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9fcf84-6c39-4b04-90a5-8a3bd83caa0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:21:22.968884 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9fcf84-6c39-4b04-90a5-8a3bd83caa0a admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:21:22.969167 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9fcf84-6c39-4b04-90a5-8a3bd83caa0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:21:22.969439 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9fcf84-6c39-4b04-90a5-8a3bd83caa0a admin admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:21:22.970065 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9fcf84-6c39-4b04-90a5-8a3bd83caa0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:21:22.972006 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9fcf84-6c39-4b04-90a5-8a3bd83caa0a admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:21:23.128696 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1282/2564] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:21:22 2022] POST /identity/v3/users => generated 289 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:21:23.132815 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5acb1d-f5b0-40c2-9764-996a7c9bb40a admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:21:23.143656 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5acb1d-f5b0-40c2-9764-996a7c9bb40a admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:21:23.148131 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30381807-c4ff-4cf9-941a-deb9d609b810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:21:23.148758 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30381807-c4ff-4cf9-941a-deb9d609b810 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:21:23.149037 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30381807-c4ff-4cf9-941a-deb9d609b810 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:21:23.149316 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30381807-c4ff-4cf9-941a-deb9d609b810 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:21:23.150071 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30381807-c4ff-4cf9-941a-deb9d609b810 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:21:23.151667 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30381807-c4ff-4cf9-941a-deb9d609b810 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:21:23.161302 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1283/2565] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:21:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:21:23.165260 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9fcf84-6c39-4b04-90a5-8a3bd83caa0a admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:21:23.175170 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9fcf84-6c39-4b04-90a5-8a3bd83caa0a admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:21:23.179873 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5c96b7-d4f1-4c19-82bd-7c2e26efba17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:21:23.180487 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c96b7-d4f1-4c19-82bd-7c2e26efba17 admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:21:23.182951 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c96b7-d4f1-4c19-82bd-7c2e26efba17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:21:23.182951 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c96b7-d4f1-4c19-82bd-7c2e26efba17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:21:23.182951 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5c96b7-d4f1-4c19-82bd-7c2e26efba17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:21:23.183897 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5c96b7-d4f1-4c19-82bd-7c2e26efba17 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:21:23.194558 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1283/2566] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:21:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:21:23.199616 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30381807-c4ff-4cf9-941a-deb9d609b810 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:21:23.209307 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30381807-c4ff-4cf9-941a-deb9d609b810 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:21:23.213790 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710cfe1e-bb87-4b94-bf96-b3b7777e87d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:21:23.214452 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710cfe1e-bb87-4b94-bf96-b3b7777e87d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:21:23.214729 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710cfe1e-bb87-4b94-bf96-b3b7777e87d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:21:23.214996 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710cfe1e-bb87-4b94-bf96-b3b7777e87d2 admin admin] PATH_INFO: `/v3/projects/8168ba3e30a743a582d1c182ca2423b3/users/2dae6a7ec4404626b75a6702386d091b/roles/56e4a96094e9434cbbafb22579c034f5` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:21:23.229719 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710cfe1e-bb87-4b94-bf96-b3b7777e87d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8168ba3e30a743a582d1c182ca2423b3, user_id=2dae6a7ec4404626b75a6702386d091b, role_id=56e4a96094e9434cbbafb22579c034f5)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:21:23.232395 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710cfe1e-bb87-4b94-bf96-b3b7777e87d2 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:21:23.330649 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1284/2567] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:21:23 2022] PUT /identity/v3/projects/8168ba3e30a743a582d1c182ca2423b3/users/2dae6a7ec4404626b75a6702386d091b/roles/56e4a96094e9434cbbafb22579c034f5 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:21:23.335664 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5537a0e9-d096-4af6-b7d1-a53e8e7fb29b None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:21:23.340665 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5537a0e9-d096-4af6-b7d1-a53e8e7fb29b None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:21:23.340665 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5537a0e9-d096-4af6-b7d1-a53e8e7fb29b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:21:23.355515 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-5537a0e9-d096-4af6-b7d1-a53e8e7fb29b None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:21:23.357063 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-5537a0e9-d096-4af6-b7d1-a53e8e7fb29b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:21:23.415463 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1284/2568] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:21:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:21:23.425078 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710cfe1e-bb87-4b94-bf96-b3b7777e87d2 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:21:23.426756 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-710cfe1e-bb87-4b94-bf96-b3b7777e87d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:21:23.441641 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:21:23.484363 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710cfe1e-bb87-4b94-bf96-b3b7777e87d2 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:21:23.487974 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82f644b-41f0-4d49-953e-c0fd2d64e7b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:21:23.491086 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82f644b-41f0-4d49-953e-c0fd2d64e7b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:21:23.491086 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82f644b-41f0-4d49-953e-c0fd2d64e7b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:21:23.491086 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82f644b-41f0-4d49-953e-c0fd2d64e7b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:21:23.498631 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82f644b-41f0-4d49-953e-c0fd2d64e7b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:21:23.500093 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82f644b-41f0-4d49-953e-c0fd2d64e7b2 service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:21:23.509561 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1285/2569] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:21:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:21:28.943609 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa03a972-3b41-44ca-a625-7a3fda1bab0a None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:21:28.944165 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa03a972-3b41-44ca-a625-7a3fda1bab0a None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:21:28.944165 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa03a972-3b41-44ca-a625-7a3fda1bab0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:21:28.976593 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-aa03a972-3b41-44ca-a625-7a3fda1bab0a None None] MFA Rules not processed for user `2dae6a7ec4404626b75a6702386d091b`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:21:28.978700 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-aa03a972-3b41-44ca-a625-7a3fda1bab0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:21:29.057547 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1285/2570] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:21:28 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:21:29.062063 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82f644b-41f0-4d49-953e-c0fd2d64e7b2 service neutron] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:21:29.072372 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82f644b-41f0-4d49-953e-c0fd2d64e7b2 service neutron] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:21:29.077276 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b044b9-0f52-46bb-b1ba-12960e5f98b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:21:29.077890 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b044b9-0f52-46bb-b1ba-12960e5f98b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:21:29.078171 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b044b9-0f52-46bb-b1ba-12960e5f98b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:21:29.078443 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b044b9-0f52-46bb-b1ba-12960e5f98b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:21:29.079295 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b044b9-0f52-46bb-b1ba-12960e5f98b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:21:29.081513 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b044b9-0f52-46bb-b1ba-12960e5f98b9 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:21:29.172955 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1286/2571] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:21:29 2022] POST /identity/v3/projects => generated 379 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 22:21:29.177156 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa03a972-3b41-44ca-a625-7a3fda1bab0a None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:21:29.188374 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa03a972-3b41-44ca-a625-7a3fda1bab0a None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:21:29.240367 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:21:29.247163 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838e0b1d-7c63-45ac-b487-6d66857eeaa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:21:29.250522 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e0b1d-7c63-45ac-b487-6d66857eeaa1 admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:21:29.250522 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e0b1d-7c63-45ac-b487-6d66857eeaa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:21:29.250522 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e0b1d-7c63-45ac-b487-6d66857eeaa1 admin admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:21:29.250522 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838e0b1d-7c63-45ac-b487-6d66857eeaa1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:21:29.251356 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838e0b1d-7c63-45ac-b487-6d66857eeaa1 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:21:29.324850 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1286/2572] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:21:29 2022] POST /identity/v3/users => generated 296 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 22:21:29.328848 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b044b9-0f52-46bb-b1ba-12960e5f98b9 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:21:29.340029 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b044b9-0f52-46bb-b1ba-12960e5f98b9 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:21:29.344746 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e381ad1f-2c32-45f7-a4b9-d5f361ef17c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:21:29.348194 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e381ad1f-2c32-45f7-a4b9-d5f361ef17c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:21:29.348194 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e381ad1f-2c32-45f7-a4b9-d5f361ef17c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:21:29.348194 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e381ad1f-2c32-45f7-a4b9-d5f361ef17c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:21:29.348194 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e381ad1f-2c32-45f7-a4b9-d5f361ef17c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:21:29.348606 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e381ad1f-2c32-45f7-a4b9-d5f361ef17c2 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:21:29.360655 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1287/2573] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:21:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:21:29.365007 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838e0b1d-7c63-45ac-b487-6d66857eeaa1 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:21:29.375121 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838e0b1d-7c63-45ac-b487-6d66857eeaa1 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:21:29.380005 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1325dd-1822-404e-85e7-f1157db665f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:21:29.380726 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1325dd-1822-404e-85e7-f1157db665f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:21:29.381008 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1325dd-1822-404e-85e7-f1157db665f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:21:29.381284 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1325dd-1822-404e-85e7-f1157db665f7 admin admin] PATH_INFO: `/v3/projects/dc93cf73f8474c6b96faa59cfd61c850/users/3666a532a7a04fde9c3ff59d894556d4/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:21:29.397141 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1325dd-1822-404e-85e7-f1157db665f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc93cf73f8474c6b96faa59cfd61c850, user_id=3666a532a7a04fde9c3ff59d894556d4, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:21:29.400451 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1325dd-1822-404e-85e7-f1157db665f7 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:21:29.462539 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1287/2574] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:21:29 2022] PUT /identity/v3/projects/dc93cf73f8474c6b96faa59cfd61c850/users/3666a532a7a04fde9c3ff59d894556d4/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:21:29.466677 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e381ad1f-2c32-45f7-a4b9-d5f361ef17c2 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:21:29.477801 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e381ad1f-2c32-45f7-a4b9-d5f361ef17c2 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:21:29.540687 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa49103-007e-45b9-8092-38745897c3f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:21:29.541304 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa49103-007e-45b9-8092-38745897c3f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:21:29.541586 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa49103-007e-45b9-8092-38745897c3f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:21:29.541856 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa49103-007e-45b9-8092-38745897c3f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:21:29.542490 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa49103-007e-45b9-8092-38745897c3f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:21:29.544092 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa49103-007e-45b9-8092-38745897c3f5 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:21:29.554581 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1288/2575] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:21:29 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:21:29.558902 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1325dd-1822-404e-85e7-f1157db665f7 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:21:29.569178 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1325dd-1822-404e-85e7-f1157db665f7 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:21:29.574542 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807c5dc7-4499-437a-b96f-8bc27af5d74c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:21:29.575200 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c5dc7-4499-437a-b96f-8bc27af5d74c admin admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:21:29.575480 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c5dc7-4499-437a-b96f-8bc27af5d74c admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:21:29.575870 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c5dc7-4499-437a-b96f-8bc27af5d74c admin admin] PATH_INFO: `/v3/projects/dc93cf73f8474c6b96faa59cfd61c850/users/3666a532a7a04fde9c3ff59d894556d4/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:21:29.577832 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807c5dc7-4499-437a-b96f-8bc27af5d74c admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc93cf73f8474c6b96faa59cfd61c850, user_id=3666a532a7a04fde9c3ff59d894556d4, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:21:29.580498 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807c5dc7-4499-437a-b96f-8bc27af5d74c admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:21:29.595036 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1288/2576] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:21:29 2022] PUT /identity/v3/projects/dc93cf73f8474c6b96faa59cfd61c850/users/3666a532a7a04fde9c3ff59d894556d4/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 22:21:35.025312 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9d07cf-10c7-4067-a1fc-9d3e84a36150 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:21:35.025818 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9d07cf-10c7-4067-a1fc-9d3e84a36150 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:21:35.025924 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9d07cf-10c7-4067-a1fc-9d3e84a36150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:21:35.066329 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-3b9d07cf-10c7-4067-a1fc-9d3e84a36150 None None] MFA Rules not processed for user `3666a532a7a04fde9c3ff59d894556d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:21:35.067529 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-3b9d07cf-10c7-4067-a1fc-9d3e84a36150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:21:35.142991 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1289/2577] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:21:35 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 22:21:35.162631 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807c5dc7-4499-437a-b96f-8bc27af5d74c admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:21:35.165545 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-807c5dc7-4499-437a-b96f-8bc27af5d74c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:21:35.244699 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807c5dc7-4499-437a-b96f-8bc27af5d74c admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:21:35.250159 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d76efd5-3d5e-4bf5-a31e-e0190ce7a6d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a1319d33e9144859c6010843be312a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:21:35.250773 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d76efd5-3d5e-4bf5-a31e-e0190ce7a6d3 service nova] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:21:35.251047 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d76efd5-3d5e-4bf5-a31e-e0190ce7a6d3 service nova] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:21:35.251315 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d76efd5-3d5e-4bf5-a31e-e0190ce7a6d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:21:35.313742 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d76efd5-3d5e-4bf5-a31e-e0190ce7a6d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:21:35.315325 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d76efd5-3d5e-4bf5-a31e-e0190ce7a6d3 service nova] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:21:35.324690 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1289/2578] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:21:35 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:21:35.981626 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9d07cf-10c7-4067-a1fc-9d3e84a36150 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:21:35.983634 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-3b9d07cf-10c7-4067-a1fc-9d3e84a36150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:21:35.990419 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:21:36.046462 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9d07cf-10c7-4067-a1fc-9d3e84a36150 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:21:36.050610 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af416384-898d-4f40-9a42-c7089107e43f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:21:36.051346 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af416384-898d-4f40-9a42-c7089107e43f service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:21:36.051627 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af416384-898d-4f40-9a42-c7089107e43f service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:21:36.051901 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af416384-898d-4f40-9a42-c7089107e43f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:21:36.053834 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-af416384-898d-4f40-9a42-c7089107e43f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:21:36.064866 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af416384-898d-4f40-9a42-c7089107e43f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:21:36.066315 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af416384-898d-4f40-9a42-c7089107e43f service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:21:36.075892 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1290/2579] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:21:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:21:45.320282 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=75,keystone:INSERT=5 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:21:46.130089 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=24 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:21:56.638361 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d76efd5-3d5e-4bf5-a31e-e0190ce7a6d3 service nova] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:21:56.650657 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d76efd5-3d5e-4bf5-a31e-e0190ce7a6d3 service nova] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:21:56.657979 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2d337d-8df5-4ddc-817f-a366985cfd3f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a1319d33e9144859c6010843be312a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:21:56.659182 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2d337d-8df5-4ddc-817f-a366985cfd3f service nova] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:21:56.659528 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2d337d-8df5-4ddc-817f-a366985cfd3f service nova] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:21:56.659903 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2d337d-8df5-4ddc-817f-a366985cfd3f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:21:56.661856 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-df2d337d-8df5-4ddc-817f-a366985cfd3f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:21:56.675806 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2d337d-8df5-4ddc-817f-a366985cfd3f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:21:56.677513 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2d337d-8df5-4ddc-817f-a366985cfd3f service nova] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:21:56.688949 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1290/2580] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:21:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:22:01.311961 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74bf884-cb51-4ebc-833e-93a8d0e61b47 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:22:01.312490 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74bf884-cb51-4ebc-833e-93a8d0e61b47 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:22:01.312591 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74bf884-cb51-4ebc-833e-93a8d0e61b47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:22:01.336563 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-d74bf884-cb51-4ebc-833e-93a8d0e61b47 None None] MFA Rules not processed for user `2dae6a7ec4404626b75a6702386d091b`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:22:01.337784 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-d74bf884-cb51-4ebc-833e-93a8d0e61b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:22:01.344961 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1291/2581] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:22:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:22:03.447237 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2d337d-8df5-4ddc-817f-a366985cfd3f service nova] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:22:03.457524 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2d337d-8df5-4ddc-817f-a366985cfd3f service nova] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:22:03.461167 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf869d8f-d84c-4d17-a0bb-b81eccb4bcdc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:22:03.461833 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf869d8f-d84c-4d17-a0bb-b81eccb4bcdc service neutron] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:22:03.462029 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf869d8f-d84c-4d17-a0bb-b81eccb4bcdc service neutron] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:22:03.462295 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf869d8f-d84c-4d17-a0bb-b81eccb4bcdc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:22:03.463994 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-cf869d8f-d84c-4d17-a0bb-b81eccb4bcdc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:22:03.530070 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf869d8f-d84c-4d17-a0bb-b81eccb4bcdc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:22:03.531505 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf869d8f-d84c-4d17-a0bb-b81eccb4bcdc service neutron] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:22:03.541535 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1291/2582] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:22:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:22:09.088390 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74bf884-cb51-4ebc-833e-93a8d0e61b47 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:22:09.090057 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-d74bf884-cb51-4ebc-833e-93a8d0e61b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:22:09.109403 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74bf884-cb51-4ebc-833e-93a8d0e61b47 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:22:09.113948 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07a44c4-c3a6-4cb0-9efc-a4f236132b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:22:09.114703 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07a44c4-c3a6-4cb0-9efc-a4f236132b2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:22:09.115088 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07a44c4-c3a6-4cb0-9efc-a4f236132b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:22:09.115457 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07a44c4-c3a6-4cb0-9efc-a4f236132b2d admin admin] PATH_INFO: `/v3/users/dfc4de8c76d0470a892dc8bad4d29441` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:22:09.117098 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07a44c4-c3a6-4cb0-9efc-a4f236132b2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=dfc4de8c76d0470a892dc8bad4d29441)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:22:09.119781 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07a44c4-c3a6-4cb0-9efc-a4f236132b2d admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:22:09.289749 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-a07a44c4-c3a6-4cb0-9efc-a4f236132b2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dfc4de8c76d0470a892dc8bad4d29441'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:22:09.305161 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-a07a44c4-c3a6-4cb0-9efc-a4f236132b2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dfc4de8c76d0470a892dc8bad4d29441'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:22:09.312371 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-a07a44c4-c3a6-4cb0-9efc-a4f236132b2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dfc4de8c76d0470a892dc8bad4d29441'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:22:09.313131 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-a07a44c4-c3a6-4cb0-9efc-a4f236132b2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dfc4de8c76d0470a892dc8bad4d29441'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:22:09.313761 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-a07a44c4-c3a6-4cb0-9efc-a4f236132b2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dfc4de8c76d0470a892dc8bad4d29441'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:22:09.397528 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1292/2583] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:22:09 2022] DELETE /identity/v3/users/dfc4de8c76d0470a892dc8bad4d29441 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:22:09.642241 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf869d8f-d84c-4d17-a0bb-b81eccb4bcdc service neutron] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:22:09.655468 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf869d8f-d84c-4d17-a0bb-b81eccb4bcdc service neutron] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:22:09.728877 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112e52a7-a256-4f6a-b9f1-c61a9ea63dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:22:09.729511 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112e52a7-a256-4f6a-b9f1-c61a9ea63dc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:22:09.729785 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112e52a7-a256-4f6a-b9f1-c61a9ea63dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:22:09.730053 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112e52a7-a256-4f6a-b9f1-c61a9ea63dc1 admin admin] PATH_INFO: `/v3/projects/8e000f22d8284f6ea2178aa17dc70151` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:22:09.731264 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112e52a7-a256-4f6a-b9f1-c61a9ea63dc1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e000f22d8284f6ea2178aa17dc70151)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:22:09.733254 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112e52a7-a256-4f6a-b9f1-c61a9ea63dc1 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:22:09.876511 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-112e52a7-a256-4f6a-b9f1-c61a9ea63dc1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e000f22d8284f6ea2178aa17dc70151', 'request_id': 'req-112e52a7-a256-4f6a-b9f1-c61a9ea63dc1'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:22:09.961197 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-112e52a7-a256-4f6a-b9f1-c61a9ea63dc1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e000f22d8284f6ea2178aa17dc70151', 'request_id': 'req-112e52a7-a256-4f6a-b9f1-c61a9ea63dc1'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:22:09.967039 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-112e52a7-a256-4f6a-b9f1-c61a9ea63dc1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e000f22d8284f6ea2178aa17dc70151', 'request_id': 'req-112e52a7-a256-4f6a-b9f1-c61a9ea63dc1'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:22:09.979065 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-112e52a7-a256-4f6a-b9f1-c61a9ea63dc1 admin admin] The token cache is being invalidate because project 8e000f22d8284f6ea2178aa17dc70151 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:22:09.979350 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-112e52a7-a256-4f6a-b9f1-c61a9ea63dc1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:22:09.979980 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-112e52a7-a256-4f6a-b9f1-c61a9ea63dc1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:22:09.981501 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1292/2584] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:22:09 2022] DELETE /identity/v3/projects/8e000f22d8284f6ea2178aa17dc70151 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:22:09.987260 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f051868-83e6-4160-85d7-c6a8f0137bfc None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:22:09.987550 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f051868-83e6-4160-85d7-c6a8f0137bfc None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:22:09.987829 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f051868-83e6-4160-85d7-c6a8f0137bfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:22:10.008768 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-0f051868-83e6-4160-85d7-c6a8f0137bfc None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:22:10.010017 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-0f051868-83e6-4160-85d7-c6a8f0137bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:22:10.068963 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1293/2585] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:22:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:22:10.077562 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22933f7-ba2a-4d62-85b0-ef66b4f07678 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:22:10.077841 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22933f7-ba2a-4d62-85b0-ef66b4f07678 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:22:10.078111 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22933f7-ba2a-4d62-85b0-ef66b4f07678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:22:10.098717 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-a22933f7-ba2a-4d62-85b0-ef66b4f07678 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:22:10.099945 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-a22933f7-ba2a-4d62-85b0-ef66b4f07678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:22:10.108300 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1293/2586] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:22:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:22:10.113443 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f051868-83e6-4160-85d7-c6a8f0137bfc None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:22:10.123190 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f051868-83e6-4160-85d7-c6a8f0137bfc None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:22:10.128097 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4110c4e6-9556-4248-a34c-f6a20c2f5ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:22:10.128673 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4110c4e6-9556-4248-a34c-f6a20c2f5ebc admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:22:10.128952 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4110c4e6-9556-4248-a34c-f6a20c2f5ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:22:10.129220 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4110c4e6-9556-4248-a34c-f6a20c2f5ebc admin admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:22:10.129880 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4110c4e6-9556-4248-a34c-f6a20c2f5ebc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 22:22:10.130175 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4110c4e6-9556-4248-a34c-f6a20c2f5ebc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:22:10.131823 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4110c4e6-9556-4248-a34c-f6a20c2f5ebc admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:22:10.143527 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1294/2587] 10.222.0.18 () {58 vars in 1153 bytes} [Sat Aug 6 22:22:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 22:22:10.147809 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22933f7-ba2a-4d62-85b0-ef66b4f07678 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:22:10.153391 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:22:10.157302 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22933f7-ba2a-4d62-85b0-ef66b4f07678 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:22:10.164485 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e215c34d-f092-4979-acde-fef097a82c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:22:10.164485 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e215c34d-f092-4979-acde-fef097a82c96 admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:22:10.164485 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e215c34d-f092-4979-acde-fef097a82c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:22:10.164485 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e215c34d-f092-4979-acde-fef097a82c96 admin admin] PATH_INFO: `/v3/projects` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:22:10.165009 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e215c34d-f092-4979-acde-fef097a82c96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:22:10.166362 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e215c34d-f092-4979-acde-fef097a82c96 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:22:10.285412 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1294/2588] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:22:10 2022] POST /identity/v3/projects => generated 375 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:22:10.289504 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4110c4e6-9556-4248-a34c-f6a20c2f5ebc admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:22:10.300332 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4110c4e6-9556-4248-a34c-f6a20c2f5ebc admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:22:10.347530 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:22:10.358930 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f68b1f-8292-46cc-bdd5-943b1074f5ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:22:10.358930 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f68b1f-8292-46cc-bdd5-943b1074f5ef admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:22:10.358930 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f68b1f-8292-46cc-bdd5-943b1074f5ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:22:10.358930 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f68b1f-8292-46cc-bdd5-943b1074f5ef admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:22:10.359486 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f68b1f-8292-46cc-bdd5-943b1074f5ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:22:10.360039 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f68b1f-8292-46cc-bdd5-943b1074f5ef admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:22:10.487537 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1295/2589] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:22:10 2022] POST /identity/v3/users => generated 288 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 22:22:10.491792 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e215c34d-f092-4979-acde-fef097a82c96 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:22:10.502227 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e215c34d-f092-4979-acde-fef097a82c96 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:22:10.506797 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48958719-b5fc-4720-a0da-b1231c725a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:22:10.507391 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48958719-b5fc-4720-a0da-b1231c725a45 admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:22:10.507671 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48958719-b5fc-4720-a0da-b1231c725a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:22:10.507942 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48958719-b5fc-4720-a0da-b1231c725a45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:22:10.508685 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48958719-b5fc-4720-a0da-b1231c725a45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:22:10.510359 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48958719-b5fc-4720-a0da-b1231c725a45 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:22:10.520888 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1295/2590] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:22:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:22:10.524891 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f68b1f-8292-46cc-bdd5-943b1074f5ef admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:22:10.535331 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f68b1f-8292-46cc-bdd5-943b1074f5ef admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:22:10.539901 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3d1d63-640c-412f-babf-1cd860fab300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:22:10.540516 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d1d63-640c-412f-babf-1cd860fab300 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:22:10.540798 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d1d63-640c-412f-babf-1cd860fab300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:22:10.541073 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d1d63-640c-412f-babf-1cd860fab300 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:22:10.541759 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3d1d63-640c-412f-babf-1cd860fab300 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:22:10.543295 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3d1d63-640c-412f-babf-1cd860fab300 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:22:10.553617 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1296/2591] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:22:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:22:10.557817 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48958719-b5fc-4720-a0da-b1231c725a45 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:22:10.568358 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48958719-b5fc-4720-a0da-b1231c725a45 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:22:10.573410 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab71903f-943d-4afd-a917-54dd0518edf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:22:10.574053 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab71903f-943d-4afd-a917-54dd0518edf6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:22:10.574334 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab71903f-943d-4afd-a917-54dd0518edf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:22:10.575184 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab71903f-943d-4afd-a917-54dd0518edf6 admin admin] PATH_INFO: `/v3/projects/8abd60d3224e4e2995d55417a73ffe58/users/563808cf212c40e2bd7f3ace5d1aec03/roles/56e4a96094e9434cbbafb22579c034f5` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:22:10.589753 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab71903f-943d-4afd-a917-54dd0518edf6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8abd60d3224e4e2995d55417a73ffe58, user_id=563808cf212c40e2bd7f3ace5d1aec03, role_id=56e4a96094e9434cbbafb22579c034f5)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:22:10.592516 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab71903f-943d-4afd-a917-54dd0518edf6 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:22:10.689111 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1296/2592] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:22:10 2022] PUT /identity/v3/projects/8abd60d3224e4e2995d55417a73ffe58/users/563808cf212c40e2bd7f3ace5d1aec03/roles/56e4a96094e9434cbbafb22579c034f5 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:22:10.694139 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513d60c1-ae4a-49b0-93f3-ffc1821f7a9d None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:22:10.694452 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513d60c1-ae4a-49b0-93f3-ffc1821f7a9d None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:22:10.694723 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513d60c1-ae4a-49b0-93f3-ffc1821f7a9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:22:10.713580 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-513d60c1-ae4a-49b0-93f3-ffc1821f7a9d None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:22:10.714769 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-513d60c1-ae4a-49b0-93f3-ffc1821f7a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:22:10.776346 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1297/2593] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:22:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 22:22:10.786469 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab71903f-943d-4afd-a917-54dd0518edf6 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:22:10.788282 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-ab71903f-943d-4afd-a917-54dd0518edf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:22:10.859395 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab71903f-943d-4afd-a917-54dd0518edf6 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:22:10.866027 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680aa101-ce2c-401b-a382-038c33782d86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:22:10.866668 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680aa101-ce2c-401b-a382-038c33782d86 service neutron] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:22:10.867052 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680aa101-ce2c-401b-a382-038c33782d86 service neutron] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:22:10.867533 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680aa101-ce2c-401b-a382-038c33782d86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:22:10.879565 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680aa101-ce2c-401b-a382-038c33782d86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:22:10.881032 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680aa101-ce2c-401b-a382-038c33782d86 service neutron] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:22:10.894368 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1297/2594] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:22:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:22:13.740030 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513d60c1-ae4a-49b0-93f3-ffc1821f7a9d None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:22:13.749810 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513d60c1-ae4a-49b0-93f3-ffc1821f7a9d None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:22:13.755089 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14db32b8-0a87-4ae5-9de9-ba05a35fbb9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:22:13.755761 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14db32b8-0a87-4ae5-9de9-ba05a35fbb9a service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:22:13.756259 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14db32b8-0a87-4ae5-9de9-ba05a35fbb9a service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:22:13.758168 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14db32b8-0a87-4ae5-9de9-ba05a35fbb9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:22:13.761456 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-14db32b8-0a87-4ae5-9de9-ba05a35fbb9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:22:13.784859 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14db32b8-0a87-4ae5-9de9-ba05a35fbb9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:22:13.788079 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14db32b8-0a87-4ae5-9de9-ba05a35fbb9a service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:22:13.807537 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1298/2595] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:22:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:22:17.540793 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38ef5b1-e13a-4064-8c6a-6b03aa7e2f3c None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:22:17.541847 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38ef5b1-e13a-4064-8c6a-6b03aa7e2f3c None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:22:17.542058 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38ef5b1-e13a-4064-8c6a-6b03aa7e2f3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:22:17.594227 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-e38ef5b1-e13a-4064-8c6a-6b03aa7e2f3c None None] MFA Rules not processed for user `563808cf212c40e2bd7f3ace5d1aec03`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:22:17.595587 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-e38ef5b1-e13a-4064-8c6a-6b03aa7e2f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:22:17.679297 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1298/2596] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:22:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:22:17.694987 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14db32b8-0a87-4ae5-9de9-ba05a35fbb9a service neutron] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:22:17.698391 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-14db32b8-0a87-4ae5-9de9-ba05a35fbb9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:22:17.799224 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14db32b8-0a87-4ae5-9de9-ba05a35fbb9a service neutron] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:22:17.806221 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d55a781-8498-466b-8771-89b812ed842d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a1319d33e9144859c6010843be312a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:22:17.806842 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d55a781-8498-466b-8771-89b812ed842d service nova] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:22:17.807124 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d55a781-8498-466b-8771-89b812ed842d service nova] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:22:17.807438 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d55a781-8498-466b-8771-89b812ed842d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:22:17.817093 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d55a781-8498-466b-8771-89b812ed842d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:22:17.818697 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d55a781-8498-466b-8771-89b812ed842d service nova] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:22:17.836362 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1299/2597] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:22:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:22:18.500722 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38ef5b1-e13a-4064-8c6a-6b03aa7e2f3c None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:22:18.509469 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-e38ef5b1-e13a-4064-8c6a-6b03aa7e2f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:22:18.534248 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38ef5b1-e13a-4064-8c6a-6b03aa7e2f3c None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:22:18.544900 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b282b744-c023-42a1-9061-275df41bca5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:22:18.547398 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b282b744-c023-42a1-9061-275df41bca5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:22:18.548425 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b282b744-c023-42a1-9061-275df41bca5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:22:18.549947 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b282b744-c023-42a1-9061-275df41bca5f admin admin] PATH_INFO: `/v3/users/0d6e35038fed4b44b986902a5ef55ce8` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:22:18.553350 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b282b744-c023-42a1-9061-275df41bca5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d6e35038fed4b44b986902a5ef55ce8)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:22:18.557436 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b282b744-c023-42a1-9061-275df41bca5f admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:22:18.826296 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:22:18.836310 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-b282b744-c023-42a1-9061-275df41bca5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d6e35038fed4b44b986902a5ef55ce8'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:22:18.850881 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-b282b744-c023-42a1-9061-275df41bca5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d6e35038fed4b44b986902a5ef55ce8'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:22:19.012898 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-b282b744-c023-42a1-9061-275df41bca5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d6e35038fed4b44b986902a5ef55ce8'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:22:19.021704 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-b282b744-c023-42a1-9061-275df41bca5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d6e35038fed4b44b986902a5ef55ce8'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:22:19.022391 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-b282b744-c023-42a1-9061-275df41bca5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d6e35038fed4b44b986902a5ef55ce8'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:22:19.028257 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1299/2598] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:22:18 2022] DELETE /identity/v3/users/0d6e35038fed4b44b986902a5ef55ce8 => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:22:19.289319 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d55a781-8498-466b-8771-89b812ed842d service nova] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:22:19.300495 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d55a781-8498-466b-8771-89b812ed842d service nova] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:22:19.355952 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:22:19.382633 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc5bf05-b1ce-4584-9bba-5a5dd2258f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:22:19.383285 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc5bf05-b1ce-4584-9bba-5a5dd2258f54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:22:19.383565 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc5bf05-b1ce-4584-9bba-5a5dd2258f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:22:19.383836 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc5bf05-b1ce-4584-9bba-5a5dd2258f54 admin admin] PATH_INFO: `/v3/projects/f4867885fc0d4ca29d2d674465398466` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:22:19.385062 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc5bf05-b1ce-4584-9bba-5a5dd2258f54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4867885fc0d4ca29d2d674465398466)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:22:19.387620 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc5bf05-b1ce-4584-9bba-5a5dd2258f54 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:22:19.473972 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b282b744-c023-42a1-9061-275df41bca5f admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:22:19.475763 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-b282b744-c023-42a1-9061-275df41bca5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:22:19.540116 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-7cc5bf05-b1ce-4584-9bba-5a5dd2258f54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4867885fc0d4ca29d2d674465398466', 'request_id': 'req-7cc5bf05-b1ce-4584-9bba-5a5dd2258f54'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:22:19.568842 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b282b744-c023-42a1-9061-275df41bca5f admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:22:19.619757 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-7cc5bf05-b1ce-4584-9bba-5a5dd2258f54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4867885fc0d4ca29d2d674465398466', 'request_id': 'req-7cc5bf05-b1ce-4584-9bba-5a5dd2258f54'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:22:19.628604 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-7cc5bf05-b1ce-4584-9bba-5a5dd2258f54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4867885fc0d4ca29d2d674465398466', 'request_id': 'req-7cc5bf05-b1ce-4584-9bba-5a5dd2258f54'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:22:19.644075 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-7cc5bf05-b1ce-4584-9bba-5a5dd2258f54 admin admin] The token cache is being invalidate because project f4867885fc0d4ca29d2d674465398466 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:22:19.644549 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-7cc5bf05-b1ce-4584-9bba-5a5dd2258f54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:22:19.645358 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-7cc5bf05-b1ce-4584-9bba-5a5dd2258f54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:22:19.647249 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1300/2599] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:22:19 2022] DELETE /identity/v3/projects/f4867885fc0d4ca29d2d674465398466 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:22:19.651735 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7803c00-e73a-433a-890a-a9d02c79e986 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:22:19.652704 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7803c00-e73a-433a-890a-a9d02c79e986 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:22:19.652704 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7803c00-e73a-433a-890a-a9d02c79e986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:22:19.680035 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-e7803c00-e73a-433a-890a-a9d02c79e986 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:22:19.681656 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-e7803c00-e73a-433a-890a-a9d02c79e986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:22:19.694514 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632b8773-c963-4be9-9d7c-823f7f1aee53 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a1319d33e9144859c6010843be312a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:22:19.695450 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632b8773-c963-4be9-9d7c-823f7f1aee53 service nova] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:22:19.696161 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632b8773-c963-4be9-9d7c-823f7f1aee53 service nova] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:22:19.696723 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632b8773-c963-4be9-9d7c-823f7f1aee53 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:22:19.703438 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-632b8773-c963-4be9-9d7c-823f7f1aee53 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:22:19.728260 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632b8773-c963-4be9-9d7c-823f7f1aee53 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:22:19.729900 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632b8773-c963-4be9-9d7c-823f7f1aee53 service nova] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:22:19.749991 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1300/2600] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:22:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 22:22:19.787959 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1301/2601] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:22:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 22:22:19.788085 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322ff3bd-414a-489c-b0f7-6cc221a929d5 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:22:19.788085 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322ff3bd-414a-489c-b0f7-6cc221a929d5 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:22:19.788085 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322ff3bd-414a-489c-b0f7-6cc221a929d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:22:19.812275 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-322ff3bd-414a-489c-b0f7-6cc221a929d5 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:22:19.814312 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-322ff3bd-414a-489c-b0f7-6cc221a929d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:22:19.827239 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1301/2602] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:22:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:22:19.828182 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7803c00-e73a-433a-890a-a9d02c79e986 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:22:19.850424 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7803c00-e73a-433a-890a-a9d02c79e986 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:22:19.854281 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcf39b9-cc57-4aeb-a3d3-34cbff247bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:22:19.855144 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcf39b9-cc57-4aeb-a3d3-34cbff247bfb admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:22:19.855604 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcf39b9-cc57-4aeb-a3d3-34cbff247bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:22:19.856084 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcf39b9-cc57-4aeb-a3d3-34cbff247bfb admin admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:22:19.857090 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcf39b9-cc57-4aeb-a3d3-34cbff247bfb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 22:22:19.857802 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcf39b9-cc57-4aeb-a3d3-34cbff247bfb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:22:19.867786 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcf39b9-cc57-4aeb-a3d3-34cbff247bfb admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:22:19.901052 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1302/2603] 10.222.0.18 () {58 vars in 1153 bytes} [Sat Aug 6 22:22:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 22:22:19.905420 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322ff3bd-414a-489c-b0f7-6cc221a929d5 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:22:19.917593 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322ff3bd-414a-489c-b0f7-6cc221a929d5 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:22:19.925847 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589898ea-6971-47f6-97d0-0954b289fadc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:22:19.926852 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589898ea-6971-47f6-97d0-0954b289fadc admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:22:19.927327 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589898ea-6971-47f6-97d0-0954b289fadc admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:22:19.927784 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589898ea-6971-47f6-97d0-0954b289fadc admin admin] PATH_INFO: `/v3/projects` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:22:19.928590 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589898ea-6971-47f6-97d0-0954b289fadc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:22:19.935213 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589898ea-6971-47f6-97d0-0954b289fadc admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:22:20.071006 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1302/2604] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:22:19 2022] POST /identity/v3/projects => generated 385 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:22:20.072925 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcf39b9-cc57-4aeb-a3d3-34cbff247bfb admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:22:20.084745 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcf39b9-cc57-4aeb-a3d3-34cbff247bfb admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:22:20.158451 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a851333-1d4e-4aea-84ea-37c9f82dd4e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:22:20.159540 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a851333-1d4e-4aea-84ea-37c9f82dd4e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:22:20.159961 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a851333-1d4e-4aea-84ea-37c9f82dd4e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:22:20.160363 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a851333-1d4e-4aea-84ea-37c9f82dd4e3 admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:22:20.161150 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a851333-1d4e-4aea-84ea-37c9f82dd4e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:22:20.164142 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a851333-1d4e-4aea-84ea-37c9f82dd4e3 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:22:20.192039 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:22:20.264164 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1303/2605] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:22:20 2022] POST /identity/v3/users => generated 293 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:22:20.265730 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589898ea-6971-47f6-97d0-0954b289fadc admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:22:20.285786 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589898ea-6971-47f6-97d0-0954b289fadc admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:22:20.291455 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed101b81-bf63-4e2c-ba09-2c5992f8045c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:22:20.292316 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed101b81-bf63-4e2c-ba09-2c5992f8045c admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:22:20.292798 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed101b81-bf63-4e2c-ba09-2c5992f8045c admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:22:20.293254 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed101b81-bf63-4e2c-ba09-2c5992f8045c admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:22:20.294057 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed101b81-bf63-4e2c-ba09-2c5992f8045c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:22:20.295825 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed101b81-bf63-4e2c-ba09-2c5992f8045c admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:22:20.298145 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:22:20.314105 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1303/2606] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:22:20 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:22:20.315683 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a851333-1d4e-4aea-84ea-37c9f82dd4e3 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:22:20.331089 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a851333-1d4e-4aea-84ea-37c9f82dd4e3 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:22:20.336808 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8271d1eb-6ebd-4218-8714-a99511fb60c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:22:20.339564 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8271d1eb-6ebd-4218-8714-a99511fb60c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:22:20.339564 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8271d1eb-6ebd-4218-8714-a99511fb60c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:22:20.339564 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8271d1eb-6ebd-4218-8714-a99511fb60c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:22:20.340283 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8271d1eb-6ebd-4218-8714-a99511fb60c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:22:20.342304 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8271d1eb-6ebd-4218-8714-a99511fb60c8 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:22:20.370111 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1304/2607] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:22:20 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:22:20.374036 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed101b81-bf63-4e2c-ba09-2c5992f8045c admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:22:20.387690 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed101b81-bf63-4e2c-ba09-2c5992f8045c admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:22:20.393376 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae298093-78ad-494c-8618-8938d872fb97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:22:20.394328 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae298093-78ad-494c-8618-8938d872fb97 admin admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:22:20.394843 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae298093-78ad-494c-8618-8938d872fb97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:22:20.395115 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae298093-78ad-494c-8618-8938d872fb97 admin admin] PATH_INFO: `/v3/projects/23e22bb596ac4d93b8b004eecc50e91e/users/f69cc7c7585048fca2dc50a4bb4d80f8/roles/56e4a96094e9434cbbafb22579c034f5` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:22:20.412355 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae298093-78ad-494c-8618-8938d872fb97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23e22bb596ac4d93b8b004eecc50e91e, user_id=f69cc7c7585048fca2dc50a4bb4d80f8, role_id=56e4a96094e9434cbbafb22579c034f5)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:22:20.415370 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae298093-78ad-494c-8618-8938d872fb97 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:22:20.579625 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1304/2608] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:22:20 2022] PUT /identity/v3/projects/23e22bb596ac4d93b8b004eecc50e91e/users/f69cc7c7585048fca2dc50a4bb4d80f8/roles/56e4a96094e9434cbbafb22579c034f5 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:22:20.591709 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2cbd17-f034-4e18-886e-71bedaf0035a None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:22:20.592211 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2cbd17-f034-4e18-886e-71bedaf0035a None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:22:20.592601 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2cbd17-f034-4e18-886e-71bedaf0035a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:22:20.627720 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-4a2cbd17-f034-4e18-886e-71bedaf0035a None None] MFA Rules not processed for user `f69cc7c7585048fca2dc50a4bb4d80f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:22:20.629049 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-4a2cbd17-f034-4e18-886e-71bedaf0035a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:22:20.704488 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1305/2609] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:22:20 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:22:20.721206 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae298093-78ad-494c-8618-8938d872fb97 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:22:20.724057 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-ae298093-78ad-494c-8618-8938d872fb97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:22:20.811376 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae298093-78ad-494c-8618-8938d872fb97 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:22:20.821071 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b37f3c-c726-4f7c-90bb-75fa8c5bb816 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a1319d33e9144859c6010843be312a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:22:20.821701 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b37f3c-c726-4f7c-90bb-75fa8c5bb816 service nova] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:22:20.821984 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b37f3c-c726-4f7c-90bb-75fa8c5bb816 service nova] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:22:20.822252 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b37f3c-c726-4f7c-90bb-75fa8c5bb816 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:22:20.836365 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b37f3c-c726-4f7c-90bb-75fa8c5bb816 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:22:20.836365 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b37f3c-c726-4f7c-90bb-75fa8c5bb816 service nova] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:22:20.847846 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1305/2610] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:22:20 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:22:21.005601 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90779d8f-0b41-4dd6-ae5c-8223b37c2f30 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:22:21.006306 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90779d8f-0b41-4dd6-ae5c-8223b37c2f30 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:22:21.006587 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90779d8f-0b41-4dd6-ae5c-8223b37c2f30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:22:21.029389 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-90779d8f-0b41-4dd6-ae5c-8223b37c2f30 None None] MFA Rules not processed for user `563808cf212c40e2bd7f3ace5d1aec03`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:22:21.030110 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-90779d8f-0b41-4dd6-ae5c-8223b37c2f30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:22:21.094339 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1306/2611] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:22:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:22:23.743086 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b37f3c-c726-4f7c-90bb-75fa8c5bb816 service nova] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:22:23.744630 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-a4b37f3c-c726-4f7c-90bb-75fa8c5bb816 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:22:23.803526 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b37f3c-c726-4f7c-90bb-75fa8c5bb816 service nova] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:22:23.807140 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe9b379-c741-4632-82a3-1c1a0fa896dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:22:23.807740 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe9b379-c741-4632-82a3-1c1a0fa896dd service neutron] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:22:23.808011 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe9b379-c741-4632-82a3-1c1a0fa896dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:22:23.808279 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe9b379-c741-4632-82a3-1c1a0fa896dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:22:23.810042 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-ffe9b379-c741-4632-82a3-1c1a0fa896dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:22:23.877907 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe9b379-c741-4632-82a3-1c1a0fa896dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:22:23.879276 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe9b379-c741-4632-82a3-1c1a0fa896dd service neutron] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:22:23.888799 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1306/2612] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:22:23 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:22:31.506266 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:INSERT=2,keystone:SELECT=64 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:22:31.638947 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90779d8f-0b41-4dd6-ae5c-8223b37c2f30 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:22:31.699344 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90779d8f-0b41-4dd6-ae5c-8223b37c2f30 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:22:31.703786 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0528d7-df15-4541-8659-763ef9a7314f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:22:31.704430 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0528d7-df15-4541-8659-763ef9a7314f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:22:31.705350 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0528d7-df15-4541-8659-763ef9a7314f admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:22:31.705632 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0528d7-df15-4541-8659-763ef9a7314f admin admin] PATH_INFO: `/v3/users/f69cc7c7585048fca2dc50a4bb4d80f8` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:22:31.706748 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0528d7-df15-4541-8659-763ef9a7314f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f69cc7c7585048fca2dc50a4bb4d80f8)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:22:31.708718 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0528d7-df15-4541-8659-763ef9a7314f admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:22:31.942868 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-0e0528d7-df15-4541-8659-763ef9a7314f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f69cc7c7585048fca2dc50a4bb4d80f8'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:22:31.955436 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-0e0528d7-df15-4541-8659-763ef9a7314f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f69cc7c7585048fca2dc50a4bb4d80f8'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:22:31.962214 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-0e0528d7-df15-4541-8659-763ef9a7314f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f69cc7c7585048fca2dc50a4bb4d80f8'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:22:31.962908 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-0e0528d7-df15-4541-8659-763ef9a7314f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f69cc7c7585048fca2dc50a4bb4d80f8'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:22:31.963662 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-0e0528d7-df15-4541-8659-763ef9a7314f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f69cc7c7585048fca2dc50a4bb4d80f8'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:22:32.048819 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1307/2613] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:22:31 2022] DELETE /identity/v3/users/f69cc7c7585048fca2dc50a4bb4d80f8 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:22:32.053521 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aa2fb4-f381-4101-b7b2-89e797bad1ae None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:22:32.054030 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aa2fb4-f381-4101-b7b2-89e797bad1ae None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:22:32.054138 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aa2fb4-f381-4101-b7b2-89e797bad1ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:22:32.068361 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:22:32.077000 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-19aa2fb4-f381-4101-b7b2-89e797bad1ae None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:22:32.077000 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-19aa2fb4-f381-4101-b7b2-89e797bad1ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:22:32.135708 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1307/2614] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:22:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 22:22:32.145447 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0528d7-df15-4541-8659-763ef9a7314f admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:22:32.146914 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-0e0528d7-df15-4541-8659-763ef9a7314f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:22:32.204556 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0528d7-df15-4541-8659-763ef9a7314f admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:22:32.208384 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41bf1da-62fa-475b-8c0c-12a807e38567 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:22:32.209009 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41bf1da-62fa-475b-8c0c-12a807e38567 service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:22:32.209288 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41bf1da-62fa-475b-8c0c-12a807e38567 service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:22:32.209557 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41bf1da-62fa-475b-8c0c-12a807e38567 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:22:32.218437 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41bf1da-62fa-475b-8c0c-12a807e38567 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:22:32.219854 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41bf1da-62fa-475b-8c0c-12a807e38567 service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:22:32.229486 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1308/2615] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:22:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:22:32.638017 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aa2fb4-f381-4101-b7b2-89e797bad1ae None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:22:32.639547 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-19aa2fb4-f381-4101-b7b2-89e797bad1ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:22:32.652960 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aa2fb4-f381-4101-b7b2-89e797bad1ae None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:22:32.658566 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66da9b3b-24e0-40fb-87e1-e26801fe5271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:22:32.659265 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66da9b3b-24e0-40fb-87e1-e26801fe5271 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:22:32.659608 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66da9b3b-24e0-40fb-87e1-e26801fe5271 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:22:32.659884 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66da9b3b-24e0-40fb-87e1-e26801fe5271 admin admin] PATH_INFO: `/v3/projects/23e22bb596ac4d93b8b004eecc50e91e` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:22:32.661097 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66da9b3b-24e0-40fb-87e1-e26801fe5271 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23e22bb596ac4d93b8b004eecc50e91e)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:22:32.662988 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66da9b3b-24e0-40fb-87e1-e26801fe5271 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:22:32.773238 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-66da9b3b-24e0-40fb-87e1-e26801fe5271 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23e22bb596ac4d93b8b004eecc50e91e', 'request_id': 'req-66da9b3b-24e0-40fb-87e1-e26801fe5271'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:22:32.840213 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-66da9b3b-24e0-40fb-87e1-e26801fe5271 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23e22bb596ac4d93b8b004eecc50e91e', 'request_id': 'req-66da9b3b-24e0-40fb-87e1-e26801fe5271'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:22:32.846725 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-66da9b3b-24e0-40fb-87e1-e26801fe5271 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23e22bb596ac4d93b8b004eecc50e91e', 'request_id': 'req-66da9b3b-24e0-40fb-87e1-e26801fe5271'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:22:32.858023 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-66da9b3b-24e0-40fb-87e1-e26801fe5271 admin admin] The token cache is being invalidate because project 23e22bb596ac4d93b8b004eecc50e91e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:22:32.858306 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-66da9b3b-24e0-40fb-87e1-e26801fe5271 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:22:32.858988 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-66da9b3b-24e0-40fb-87e1-e26801fe5271 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:22:32.860578 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1308/2616] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:22:32 2022] DELETE /identity/v3/projects/23e22bb596ac4d93b8b004eecc50e91e => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:22:32.865739 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e941489-3be4-40a9-a939-721002f1f350 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:22:32.866232 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e941489-3be4-40a9-a939-721002f1f350 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:22:32.866340 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e941489-3be4-40a9-a939-721002f1f350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:22:32.885248 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-8e941489-3be4-40a9-a939-721002f1f350 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:22:32.886493 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-8e941489-3be4-40a9-a939-721002f1f350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:22:32.903285 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:22:32.945540 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1309/2617] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:22:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:22:32.953450 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6707db04-3829-4ec2-a95f-afa1087706f4 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:22:32.953800 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6707db04-3829-4ec2-a95f-afa1087706f4 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:22:32.954077 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6707db04-3829-4ec2-a95f-afa1087706f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:22:32.975273 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-6707db04-3829-4ec2-a95f-afa1087706f4 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:22:32.976496 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-6707db04-3829-4ec2-a95f-afa1087706f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:22:32.984452 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1309/2618] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:22:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:22:32.988952 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e941489-3be4-40a9-a939-721002f1f350 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:22:32.998874 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e941489-3be4-40a9-a939-721002f1f350 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:22:33.003730 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db827f2-8b97-42e1-b0a6-64b38a833612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:22:33.004345 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db827f2-8b97-42e1-b0a6-64b38a833612 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:22:33.004648 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db827f2-8b97-42e1-b0a6-64b38a833612 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:22:33.004920 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db827f2-8b97-42e1-b0a6-64b38a833612 admin admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:22:33.005627 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db827f2-8b97-42e1-b0a6-64b38a833612 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 22:22:33.005930 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db827f2-8b97-42e1-b0a6-64b38a833612 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:22:33.007514 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db827f2-8b97-42e1-b0a6-64b38a833612 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:22:33.019476 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1310/2619] 10.222.0.18 () {58 vars in 1153 bytes} [Sat Aug 6 22:22:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 22:22:33.023930 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6707db04-3829-4ec2-a95f-afa1087706f4 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:22:33.034360 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6707db04-3829-4ec2-a95f-afa1087706f4 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:22:33.039146 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f4e386-ba3f-4787-a197-5e854197db65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:22:33.039771 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f4e386-ba3f-4787-a197-5e854197db65 admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:22:33.040051 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f4e386-ba3f-4787-a197-5e854197db65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:22:33.040322 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f4e386-ba3f-4787-a197-5e854197db65 admin admin] PATH_INFO: `/v3/projects` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:22:33.041022 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f4e386-ba3f-4787-a197-5e854197db65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:22:33.042915 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f4e386-ba3f-4787-a197-5e854197db65 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:22:33.147706 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1310/2620] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:22:33 2022] POST /identity/v3/projects => generated 371 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:22:33.151978 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db827f2-8b97-42e1-b0a6-64b38a833612 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:22:33.163037 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db827f2-8b97-42e1-b0a6-64b38a833612 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:22:33.223755 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fe0793-a220-4a05-92b7-01a557977048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:22:33.224358 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fe0793-a220-4a05-92b7-01a557977048 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:22:33.224687 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fe0793-a220-4a05-92b7-01a557977048 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:22:33.224949 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fe0793-a220-4a05-92b7-01a557977048 admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:22:33.225587 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fe0793-a220-4a05-92b7-01a557977048 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:22:33.227593 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fe0793-a220-4a05-92b7-01a557977048 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:22:33.338467 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1311/2621] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:22:33 2022] POST /identity/v3/users => generated 286 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:22:33.342689 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f4e386-ba3f-4787-a197-5e854197db65 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:22:33.355173 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f4e386-ba3f-4787-a197-5e854197db65 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:22:33.365779 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae9a606-5acb-457d-a78a-1cb88857581d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:22:33.366508 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae9a606-5acb-457d-a78a-1cb88857581d admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:22:33.366786 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae9a606-5acb-457d-a78a-1cb88857581d admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:22:33.367061 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae9a606-5acb-457d-a78a-1cb88857581d admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:22:33.367719 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae9a606-5acb-457d-a78a-1cb88857581d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:22:33.369379 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae9a606-5acb-457d-a78a-1cb88857581d admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:22:33.379791 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1311/2622] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:22:33 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:22:33.385177 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fe0793-a220-4a05-92b7-01a557977048 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:22:33.395249 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fe0793-a220-4a05-92b7-01a557977048 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:22:33.400022 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eed1a2b-bf3c-445b-a2a2-f3393ea90ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:22:33.400660 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed1a2b-bf3c-445b-a2a2-f3393ea90ca6 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:22:33.400962 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed1a2b-bf3c-445b-a2a2-f3393ea90ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:22:33.401216 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed1a2b-bf3c-445b-a2a2-f3393ea90ca6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:22:33.401852 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eed1a2b-bf3c-445b-a2a2-f3393ea90ca6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:22:33.403458 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eed1a2b-bf3c-445b-a2a2-f3393ea90ca6 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:22:33.415165 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1312/2623] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:22:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:22:33.419366 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae9a606-5acb-457d-a78a-1cb88857581d admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:22:33.430634 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae9a606-5acb-457d-a78a-1cb88857581d admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:22:33.440967 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f792763d-f600-4621-bfcd-cb3f0429626f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:22:33.440967 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f792763d-f600-4621-bfcd-cb3f0429626f admin admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:22:33.440967 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f792763d-f600-4621-bfcd-cb3f0429626f admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:22:33.440967 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f792763d-f600-4621-bfcd-cb3f0429626f admin admin] PATH_INFO: `/v3/projects/a9d56d3268064a5e8d86e9dcc3e21c86/users/2694eb5e0c7e4ad7abb1b9f2c2799472/roles/56e4a96094e9434cbbafb22579c034f5` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:22:33.452108 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f792763d-f600-4621-bfcd-cb3f0429626f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9d56d3268064a5e8d86e9dcc3e21c86, user_id=2694eb5e0c7e4ad7abb1b9f2c2799472, role_id=56e4a96094e9434cbbafb22579c034f5)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:22:33.454768 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f792763d-f600-4621-bfcd-cb3f0429626f admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:22:33.529264 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1312/2624] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:22:33 2022] PUT /identity/v3/projects/a9d56d3268064a5e8d86e9dcc3e21c86/users/2694eb5e0c7e4ad7abb1b9f2c2799472/roles/56e4a96094e9434cbbafb22579c034f5 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:22:33.540428 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08edfc7-39d6-4656-ba76-77a5fe041ace None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:22:33.540746 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08edfc7-39d6-4656-ba76-77a5fe041ace None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:22:33.541027 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08edfc7-39d6-4656-ba76-77a5fe041ace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:22:33.575383 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-d08edfc7-39d6-4656-ba76-77a5fe041ace None None] MFA Rules not processed for user `2694eb5e0c7e4ad7abb1b9f2c2799472`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:22:33.575383 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-d08edfc7-39d6-4656-ba76-77a5fe041ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:22:33.768916 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1313/2625] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:22:33 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:22:33.773487 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f792763d-f600-4621-bfcd-cb3f0429626f admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:22:33.803839 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:22:33.835253 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f792763d-f600-4621-bfcd-cb3f0429626f admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:22:33.839746 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1dd500-4867-4b40-890b-67be9d50ff22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:22:33.840349 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1dd500-4867-4b40-890b-67be9d50ff22 admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:22:33.840653 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1dd500-4867-4b40-890b-67be9d50ff22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:22:33.840989 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1dd500-4867-4b40-890b-67be9d50ff22 admin admin] PATH_INFO: `/v3/projects` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:22:33.841557 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1dd500-4867-4b40-890b-67be9d50ff22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:22:33.843428 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1dd500-4867-4b40-890b-67be9d50ff22 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:22:33.975457 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1313/2626] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:22:33 2022] POST /identity/v3/projects => generated 373 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:22:33.978073 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08edfc7-39d6-4656-ba76-77a5fe041ace None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:22:33.988956 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08edfc7-39d6-4656-ba76-77a5fe041ace None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:22:34.001044 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:22:34.045625 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be28e75-1779-4243-a543-6a43a60f2833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:22:34.046229 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be28e75-1779-4243-a543-6a43a60f2833 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:22:34.046503 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be28e75-1779-4243-a543-6a43a60f2833 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:22:34.046865 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be28e75-1779-4243-a543-6a43a60f2833 admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:22:34.047927 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be28e75-1779-4243-a543-6a43a60f2833 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:22:34.050118 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be28e75-1779-4243-a543-6a43a60f2833 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:22:34.162581 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1314/2627] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:22:33 2022] POST /identity/v3/users => generated 293 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 22:22:34.166664 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1dd500-4867-4b40-890b-67be9d50ff22 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:22:34.177165 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1dd500-4867-4b40-890b-67be9d50ff22 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:22:34.181760 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab1efff-3444-410c-a736-2e0ab6ee604d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:22:34.182345 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab1efff-3444-410c-a736-2e0ab6ee604d admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:22:34.182622 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab1efff-3444-410c-a736-2e0ab6ee604d admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:22:34.182894 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab1efff-3444-410c-a736-2e0ab6ee604d admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:22:34.183528 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab1efff-3444-410c-a736-2e0ab6ee604d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:22:34.185219 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab1efff-3444-410c-a736-2e0ab6ee604d admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:22:34.195702 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1314/2628] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:22:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 22:22:34.199735 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be28e75-1779-4243-a543-6a43a60f2833 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:22:34.210489 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be28e75-1779-4243-a543-6a43a60f2833 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:22:34.215064 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f2e957-a8c1-49b2-92cd-3f436c4a70bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:22:34.217254 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2e957-a8c1-49b2-92cd-3f436c4a70bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:22:34.217254 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2e957-a8c1-49b2-92cd-3f436c4a70bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:22:34.217254 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2e957-a8c1-49b2-92cd-3f436c4a70bf admin admin] PATH_INFO: `/v3/projects/2a770b85b7f1495586c9d377b914a61b/users/57aae112d1f84e9aa0bc12c3c357199c/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:22:34.231105 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f2e957-a8c1-49b2-92cd-3f436c4a70bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a770b85b7f1495586c9d377b914a61b, user_id=57aae112d1f84e9aa0bc12c3c357199c, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:22:34.233806 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f2e957-a8c1-49b2-92cd-3f436c4a70bf admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:22:34.338476 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1315/2629] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:22:34 2022] PUT /identity/v3/projects/2a770b85b7f1495586c9d377b914a61b/users/57aae112d1f84e9aa0bc12c3c357199c/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:22:34.342741 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab1efff-3444-410c-a736-2e0ab6ee604d admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:22:34.354285 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab1efff-3444-410c-a736-2e0ab6ee604d admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:22:34.412694 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5601b40-e898-4585-a3a3-7bdc4b976814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:22:34.413426 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5601b40-e898-4585-a3a3-7bdc4b976814 admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:22:34.413774 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5601b40-e898-4585-a3a3-7bdc4b976814 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:22:34.414115 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5601b40-e898-4585-a3a3-7bdc4b976814 admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:22:34.414913 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5601b40-e898-4585-a3a3-7bdc4b976814 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:22:34.416572 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5601b40-e898-4585-a3a3-7bdc4b976814 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:22:34.427359 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1315/2630] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:22:34 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:22:34.435908 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f2e957-a8c1-49b2-92cd-3f436c4a70bf admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:22:34.446557 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f2e957-a8c1-49b2-92cd-3f436c4a70bf admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:22:34.451005 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c213332-1c53-422c-b8bd-09875c55d0b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:22:34.451660 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c213332-1c53-422c-b8bd-09875c55d0b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:22:34.451915 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c213332-1c53-422c-b8bd-09875c55d0b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:22:34.452180 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c213332-1c53-422c-b8bd-09875c55d0b3 admin admin] PATH_INFO: `/v3/projects/2a770b85b7f1495586c9d377b914a61b/users/57aae112d1f84e9aa0bc12c3c357199c/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:22:34.454064 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c213332-1c53-422c-b8bd-09875c55d0b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a770b85b7f1495586c9d377b914a61b, user_id=57aae112d1f84e9aa0bc12c3c357199c, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:22:34.456718 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c213332-1c53-422c-b8bd-09875c55d0b3 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:22:34.466892 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1316/2631] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:22:34 2022] PUT /identity/v3/projects/2a770b85b7f1495586c9d377b914a61b/users/57aae112d1f84e9aa0bc12c3c357199c/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:22:34.478554 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301fb552-63d9-4588-b066-b5b556400d50 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:22:34.478886 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301fb552-63d9-4588-b066-b5b556400d50 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:22:34.479164 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301fb552-63d9-4588-b066-b5b556400d50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:22:34.507589 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-301fb552-63d9-4588-b066-b5b556400d50 None None] MFA Rules not processed for user `57aae112d1f84e9aa0bc12c3c357199c`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:22:34.508860 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-301fb552-63d9-4588-b066-b5b556400d50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:22:34.575858 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1316/2632] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:22:34 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:22:34.587039 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c213332-1c53-422c-b8bd-09875c55d0b3 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:22:34.588892 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-1c213332-1c53-422c-b8bd-09875c55d0b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:22:34.644158 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c213332-1c53-422c-b8bd-09875c55d0b3 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:22:34.647736 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaf14df-ce2e-4b68-88af-29a4ec978ec8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa71c71de263478da234f3c5716a6e5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:22:34.648319 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaf14df-ce2e-4b68-88af-29a4ec978ec8 service cinder] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:22:34.648618 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaf14df-ce2e-4b68-88af-29a4ec978ec8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:22:34.648886 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaf14df-ce2e-4b68-88af-29a4ec978ec8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:22:34.657719 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaf14df-ce2e-4b68-88af-29a4ec978ec8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:22:34.659436 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaf14df-ce2e-4b68-88af-29a4ec978ec8 service cinder] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:22:34.668783 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1317/2633] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:22:34 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:22:34.951194 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301fb552-63d9-4588-b066-b5b556400d50 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:22:34.961272 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301fb552-63d9-4588-b066-b5b556400d50 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:22:34.964817 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df72321f-dcdd-4a4c-ac54-b01de9f86b07 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa71c71de263478da234f3c5716a6e5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:22:34.965459 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df72321f-dcdd-4a4c-ac54-b01de9f86b07 service cinder] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:22:34.965787 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df72321f-dcdd-4a4c-ac54-b01de9f86b07 service cinder] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:22:34.966056 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df72321f-dcdd-4a4c-ac54-b01de9f86b07 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:22:34.985470 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:22:35.023219 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df72321f-dcdd-4a4c-ac54-b01de9f86b07 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:22:35.024667 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df72321f-dcdd-4a4c-ac54-b01de9f86b07 service cinder] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:22:35.033892 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1317/2634] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:22:34 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:22:44.664550 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=62,keystone:INSERT=5 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:22:45.051815 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=18 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:23:29.258452 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaf14df-ce2e-4b68-88af-29a4ec978ec8 service cinder] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:29.319137 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaf14df-ce2e-4b68-88af-29a4ec978ec8 service cinder] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:29.323453 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6a42bd-8b7f-4a4b-864d-a03806b7d680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:29.324061 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a42bd-8b7f-4a4b-864d-a03806b7d680 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:29.324341 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a42bd-8b7f-4a4b-864d-a03806b7d680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:29.324640 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a42bd-8b7f-4a4b-864d-a03806b7d680 admin admin] PATH_INFO: `/v3/users/2694eb5e0c7e4ad7abb1b9f2c2799472` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:29.325991 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6a42bd-8b7f-4a4b-864d-a03806b7d680 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2694eb5e0c7e4ad7abb1b9f2c2799472)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:29.327865 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6a42bd-8b7f-4a4b-864d-a03806b7d680 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:29.510667 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-9e6a42bd-8b7f-4a4b-864d-a03806b7d680 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2694eb5e0c7e4ad7abb1b9f2c2799472'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:29.522832 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-9e6a42bd-8b7f-4a4b-864d-a03806b7d680 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2694eb5e0c7e4ad7abb1b9f2c2799472'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:29.530552 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-9e6a42bd-8b7f-4a4b-864d-a03806b7d680 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2694eb5e0c7e4ad7abb1b9f2c2799472'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:29.531178 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-9e6a42bd-8b7f-4a4b-864d-a03806b7d680 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2694eb5e0c7e4ad7abb1b9f2c2799472'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:29.531774 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-9e6a42bd-8b7f-4a4b-864d-a03806b7d680 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2694eb5e0c7e4ad7abb1b9f2c2799472'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:29.595459 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1318/2635] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:23:29 2022] DELETE /identity/v3/users/2694eb5e0c7e4ad7abb1b9f2c2799472 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:23:29.600137 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010eed88-6e1a-434d-8ddd-1c9003ad0a68 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:29.600634 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010eed88-6e1a-434d-8ddd-1c9003ad0a68 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:29.600747 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010eed88-6e1a-434d-8ddd-1c9003ad0a68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:29.619365 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-010eed88-6e1a-434d-8ddd-1c9003ad0a68 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:23:29.621366 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-010eed88-6e1a-434d-8ddd-1c9003ad0a68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:23:29.679727 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1318/2636] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:23:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:23:29.689513 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6a42bd-8b7f-4a4b-864d-a03806b7d680 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:29.690948 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-9e6a42bd-8b7f-4a4b-864d-a03806b7d680 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:23:29.748181 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6a42bd-8b7f-4a4b-864d-a03806b7d680 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:29.751783 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ad4ff7-805c-4239-b985-4cdf24e89a3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:29.752384 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ad4ff7-805c-4239-b985-4cdf24e89a3b service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:29.752701 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ad4ff7-805c-4239-b985-4cdf24e89a3b service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:29.752971 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ad4ff7-805c-4239-b985-4cdf24e89a3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:29.761995 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ad4ff7-805c-4239-b985-4cdf24e89a3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:29.763954 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ad4ff7-805c-4239-b985-4cdf24e89a3b service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:29.773564 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1319/2637] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:23:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 22:23:30.186282 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010eed88-6e1a-434d-8ddd-1c9003ad0a68 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:30.187735 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-010eed88-6e1a-434d-8ddd-1c9003ad0a68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:23:30.199466 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010eed88-6e1a-434d-8ddd-1c9003ad0a68 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:30.205412 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d068bf-5dbf-4188-8dd0-40d7fc0a1948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:30.206134 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d068bf-5dbf-4188-8dd0-40d7fc0a1948 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:30.206436 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d068bf-5dbf-4188-8dd0-40d7fc0a1948 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:30.206708 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d068bf-5dbf-4188-8dd0-40d7fc0a1948 admin admin] PATH_INFO: `/v3/projects/a9d56d3268064a5e8d86e9dcc3e21c86` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:30.207832 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d068bf-5dbf-4188-8dd0-40d7fc0a1948 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9d56d3268064a5e8d86e9dcc3e21c86)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:30.209800 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d068bf-5dbf-4188-8dd0-40d7fc0a1948 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:30.324347 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-c9d068bf-5dbf-4188-8dd0-40d7fc0a1948 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9d56d3268064a5e8d86e9dcc3e21c86', 'request_id': 'req-c9d068bf-5dbf-4188-8dd0-40d7fc0a1948'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:30.394727 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-c9d068bf-5dbf-4188-8dd0-40d7fc0a1948 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9d56d3268064a5e8d86e9dcc3e21c86', 'request_id': 'req-c9d068bf-5dbf-4188-8dd0-40d7fc0a1948'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:30.401053 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-c9d068bf-5dbf-4188-8dd0-40d7fc0a1948 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9d56d3268064a5e8d86e9dcc3e21c86', 'request_id': 'req-c9d068bf-5dbf-4188-8dd0-40d7fc0a1948'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:30.412277 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-c9d068bf-5dbf-4188-8dd0-40d7fc0a1948 admin admin] The token cache is being invalidate because project a9d56d3268064a5e8d86e9dcc3e21c86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:23:30.412588 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-c9d068bf-5dbf-4188-8dd0-40d7fc0a1948 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:30.413219 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-c9d068bf-5dbf-4188-8dd0-40d7fc0a1948 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:30.414701 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1319/2638] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:23:30 2022] DELETE /identity/v3/projects/a9d56d3268064a5e8d86e9dcc3e21c86 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:23:30.418608 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ad4ff7-805c-4239-b985-4cdf24e89a3b service neutron] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:30.429050 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ad4ff7-805c-4239-b985-4cdf24e89a3b service neutron] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:30.447239 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:23:30.484198 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88c545a-e002-4f66-8bb4-c8d62cdf1542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:30.485958 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88c545a-e002-4f66-8bb4-c8d62cdf1542 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:30.485958 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88c545a-e002-4f66-8bb4-c8d62cdf1542 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:30.485958 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88c545a-e002-4f66-8bb4-c8d62cdf1542 admin admin] PATH_INFO: `/v3/users/57aae112d1f84e9aa0bc12c3c357199c` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:30.486533 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88c545a-e002-4f66-8bb4-c8d62cdf1542 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57aae112d1f84e9aa0bc12c3c357199c)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:30.488467 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88c545a-e002-4f66-8bb4-c8d62cdf1542 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:30.654217 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-e88c545a-e002-4f66-8bb4-c8d62cdf1542 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57aae112d1f84e9aa0bc12c3c357199c'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:30.666150 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-e88c545a-e002-4f66-8bb4-c8d62cdf1542 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57aae112d1f84e9aa0bc12c3c357199c'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:30.673753 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-e88c545a-e002-4f66-8bb4-c8d62cdf1542 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57aae112d1f84e9aa0bc12c3c357199c'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:30.674445 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-e88c545a-e002-4f66-8bb4-c8d62cdf1542 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57aae112d1f84e9aa0bc12c3c357199c'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:30.675055 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-e88c545a-e002-4f66-8bb4-c8d62cdf1542 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57aae112d1f84e9aa0bc12c3c357199c'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:30.738067 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1320/2639] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:23:30 2022] DELETE /identity/v3/users/57aae112d1f84e9aa0bc12c3c357199c => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:23:31.127869 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d068bf-5dbf-4188-8dd0-40d7fc0a1948 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:31.131580 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-c9d068bf-5dbf-4188-8dd0-40d7fc0a1948 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:23:31.143906 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d068bf-5dbf-4188-8dd0-40d7fc0a1948 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:31.202275 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354fa4fd-8c0a-45c9-8638-e097967e292a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:31.203064 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354fa4fd-8c0a-45c9-8638-e097967e292a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:31.203340 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354fa4fd-8c0a-45c9-8638-e097967e292a admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:31.203615 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354fa4fd-8c0a-45c9-8638-e097967e292a admin admin] PATH_INFO: `/v3/projects/2a770b85b7f1495586c9d377b914a61b` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:31.204768 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354fa4fd-8c0a-45c9-8638-e097967e292a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a770b85b7f1495586c9d377b914a61b)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:31.206659 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354fa4fd-8c0a-45c9-8638-e097967e292a admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:31.302607 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:23:31.318242 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-354fa4fd-8c0a-45c9-8638-e097967e292a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a770b85b7f1495586c9d377b914a61b', 'request_id': 'req-354fa4fd-8c0a-45c9-8638-e097967e292a'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:31.383407 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-354fa4fd-8c0a-45c9-8638-e097967e292a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a770b85b7f1495586c9d377b914a61b', 'request_id': 'req-354fa4fd-8c0a-45c9-8638-e097967e292a'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:31.390150 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-354fa4fd-8c0a-45c9-8638-e097967e292a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a770b85b7f1495586c9d377b914a61b', 'request_id': 'req-354fa4fd-8c0a-45c9-8638-e097967e292a'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:31.401496 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-354fa4fd-8c0a-45c9-8638-e097967e292a admin admin] The token cache is being invalidate because project 2a770b85b7f1495586c9d377b914a61b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:23:31.401781 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-354fa4fd-8c0a-45c9-8638-e097967e292a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:31.402424 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-354fa4fd-8c0a-45c9-8638-e097967e292a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:31.404742 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1320/2640] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:23:31 2022] DELETE /identity/v3/projects/2a770b85b7f1495586c9d377b914a61b => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:23:31.409853 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079de96f-25e5-4cbc-aeb2-6772d3bedf89 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:31.412324 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079de96f-25e5-4cbc-aeb2-6772d3bedf89 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:31.412324 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079de96f-25e5-4cbc-aeb2-6772d3bedf89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:31.429987 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-079de96f-25e5-4cbc-aeb2-6772d3bedf89 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:23:31.431184 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-079de96f-25e5-4cbc-aeb2-6772d3bedf89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:23:31.490331 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1321/2641] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:23:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:23:31.499943 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c873c34-1346-420b-ae12-350fbcd9c194 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:31.500229 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c873c34-1346-420b-ae12-350fbcd9c194 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:31.500531 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c873c34-1346-420b-ae12-350fbcd9c194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:31.520829 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-2c873c34-1346-420b-ae12-350fbcd9c194 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:23:31.522023 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-2c873c34-1346-420b-ae12-350fbcd9c194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:23:31.530209 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1321/2642] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:23:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:23:31.534450 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079de96f-25e5-4cbc-aeb2-6772d3bedf89 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:31.544230 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079de96f-25e5-4cbc-aeb2-6772d3bedf89 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:31.549037 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643ea70d-ab75-4a6d-8f88-f5935e33f1ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:31.549416 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643ea70d-ab75-4a6d-8f88-f5935e33f1ea admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:31.549696 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643ea70d-ab75-4a6d-8f88-f5935e33f1ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:31.549966 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643ea70d-ab75-4a6d-8f88-f5935e33f1ea admin admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:31.550643 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643ea70d-ab75-4a6d-8f88-f5935e33f1ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 22:23:31.550994 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643ea70d-ab75-4a6d-8f88-f5935e33f1ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:31.552498 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643ea70d-ab75-4a6d-8f88-f5935e33f1ea admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:31.564497 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1322/2643] 10.222.0.18 () {58 vars in 1153 bytes} [Sat Aug 6 22:23:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 22:23:31.568673 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c873c34-1346-420b-ae12-350fbcd9c194 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:31.578467 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c873c34-1346-420b-ae12-350fbcd9c194 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:31.583192 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d815a7-6830-41d2-b2b4-914f2b60346a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:31.583781 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d815a7-6830-41d2-b2b4-914f2b60346a admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:31.584054 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d815a7-6830-41d2-b2b4-914f2b60346a admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:31.584319 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d815a7-6830-41d2-b2b4-914f2b60346a admin admin] PATH_INFO: `/v3/projects` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:31.585051 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d815a7-6830-41d2-b2b4-914f2b60346a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:31.586918 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d815a7-6830-41d2-b2b4-914f2b60346a admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:31.667994 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1322/2644] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:23:31 2022] POST /identity/v3/projects => generated 383 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:23:31.672220 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643ea70d-ab75-4a6d-8f88-f5935e33f1ea admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:31.682298 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643ea70d-ab75-4a6d-8f88-f5935e33f1ea admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:31.709308 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:23:31.738608 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb51efb-8139-4a27-9be1-ddd1e1972240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:31.739201 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb51efb-8139-4a27-9be1-ddd1e1972240 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:31.739582 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb51efb-8139-4a27-9be1-ddd1e1972240 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:31.739864 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb51efb-8139-4a27-9be1-ddd1e1972240 admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:31.740883 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb51efb-8139-4a27-9be1-ddd1e1972240 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:31.743227 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb51efb-8139-4a27-9be1-ddd1e1972240 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:31.850227 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1323/2645] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:23:31 2022] POST /identity/v3/users => generated 292 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 22:23:31.854399 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d815a7-6830-41d2-b2b4-914f2b60346a admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:31.865360 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d815a7-6830-41d2-b2b4-914f2b60346a admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:31.869973 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaf95cd-678c-4ccf-b76e-a545c664355e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:31.870566 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaf95cd-678c-4ccf-b76e-a545c664355e admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:31.870846 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaf95cd-678c-4ccf-b76e-a545c664355e admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:31.871117 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaf95cd-678c-4ccf-b76e-a545c664355e admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:31.871787 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaf95cd-678c-4ccf-b76e-a545c664355e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:31.874054 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaf95cd-678c-4ccf-b76e-a545c664355e admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:31.884516 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1323/2646] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:23:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:23:31.888909 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb51efb-8139-4a27-9be1-ddd1e1972240 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:31.899098 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb51efb-8139-4a27-9be1-ddd1e1972240 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:31.904117 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cd9938-5abb-41b0-a8ab-4100c34491c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:31.905453 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd9938-5abb-41b0-a8ab-4100c34491c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:31.905453 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd9938-5abb-41b0-a8ab-4100c34491c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:31.905453 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd9938-5abb-41b0-a8ab-4100c34491c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:31.905930 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cd9938-5abb-41b0-a8ab-4100c34491c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:31.907512 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cd9938-5abb-41b0-a8ab-4100c34491c7 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:31.918011 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1324/2647] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:23:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:23:31.922871 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaf95cd-678c-4ccf-b76e-a545c664355e admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:31.933194 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaf95cd-678c-4ccf-b76e-a545c664355e admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:31.938977 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31ca0eb-0186-499b-a46d-d000e0963e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:31.939617 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31ca0eb-0186-499b-a46d-d000e0963e51 admin admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:31.939916 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31ca0eb-0186-499b-a46d-d000e0963e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:31.940185 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31ca0eb-0186-499b-a46d-d000e0963e51 admin admin] PATH_INFO: `/v3/projects/cffcd8aff30f4525bc2aa298f8c0de6f/users/ab179bc5d26c45dca8ad141c73469154/roles/56e4a96094e9434cbbafb22579c034f5` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:31.954820 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31ca0eb-0186-499b-a46d-d000e0963e51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cffcd8aff30f4525bc2aa298f8c0de6f, user_id=ab179bc5d26c45dca8ad141c73469154, role_id=56e4a96094e9434cbbafb22579c034f5)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:31.957473 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31ca0eb-0186-499b-a46d-d000e0963e51 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:32.037861 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1324/2648] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:23:31 2022] PUT /identity/v3/projects/cffcd8aff30f4525bc2aa298f8c0de6f/users/ab179bc5d26c45dca8ad141c73469154/roles/56e4a96094e9434cbbafb22579c034f5 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:23:32.050984 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1bf03a-8adb-4fa8-ac10-a8764ce0f1b6 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:32.051358 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1bf03a-8adb-4fa8-ac10-a8764ce0f1b6 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:32.051641 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1bf03a-8adb-4fa8-ac10-a8764ce0f1b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:32.079275 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-df1bf03a-8adb-4fa8-ac10-a8764ce0f1b6 None None] MFA Rules not processed for user `ab179bc5d26c45dca8ad141c73469154`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:23:32.080504 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-df1bf03a-8adb-4fa8-ac10-a8764ce0f1b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:23:32.142355 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1325/2649] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:23:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:23:32.146678 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31ca0eb-0186-499b-a46d-d000e0963e51 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:32.206368 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31ca0eb-0186-499b-a46d-d000e0963e51 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:32.210991 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4360da-f618-4ea5-be7a-2ad28b0cf166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:32.211576 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4360da-f618-4ea5-be7a-2ad28b0cf166 admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:32.211851 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4360da-f618-4ea5-be7a-2ad28b0cf166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:32.212118 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4360da-f618-4ea5-be7a-2ad28b0cf166 admin admin] PATH_INFO: `/v3/projects` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:32.212773 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4360da-f618-4ea5-be7a-2ad28b0cf166 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:32.214650 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4360da-f618-4ea5-be7a-2ad28b0cf166 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:32.303501 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1325/2650] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:23:32 2022] POST /identity/v3/projects => generated 385 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:23:32.307453 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1bf03a-8adb-4fa8-ac10-a8764ce0f1b6 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:32.318227 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1bf03a-8adb-4fa8-ac10-a8764ce0f1b6 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:32.376970 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec818ef-b5c1-4ba4-b884-a16c6a2367a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:32.377577 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec818ef-b5c1-4ba4-b884-a16c6a2367a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:32.377882 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec818ef-b5c1-4ba4-b884-a16c6a2367a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:32.378135 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec818ef-b5c1-4ba4-b884-a16c6a2367a7 admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:32.378755 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec818ef-b5c1-4ba4-b884-a16c6a2367a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:32.380700 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec818ef-b5c1-4ba4-b884-a16c6a2367a7 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:32.474783 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1326/2651] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:23:32 2022] POST /identity/v3/users => generated 299 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:23:32.481189 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4360da-f618-4ea5-be7a-2ad28b0cf166 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:32.490299 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4360da-f618-4ea5-be7a-2ad28b0cf166 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:32.494911 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7e1b4c-fe56-4fe0-b2a0-924fa7e22de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:32.495503 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e1b4c-fe56-4fe0-b2a0-924fa7e22de6 admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:32.495778 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e1b4c-fe56-4fe0-b2a0-924fa7e22de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:32.496072 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e1b4c-fe56-4fe0-b2a0-924fa7e22de6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:32.496784 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7e1b4c-fe56-4fe0-b2a0-924fa7e22de6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:32.498408 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7e1b4c-fe56-4fe0-b2a0-924fa7e22de6 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:32.508894 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1326/2652] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:23:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:23:32.521173 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec818ef-b5c1-4ba4-b884-a16c6a2367a7 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:32.531797 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec818ef-b5c1-4ba4-b884-a16c6a2367a7 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:32.536383 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8078f5-e2ea-42e5-b55b-018ce944dc9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:32.537066 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8078f5-e2ea-42e5-b55b-018ce944dc9b admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:32.537345 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8078f5-e2ea-42e5-b55b-018ce944dc9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:32.537618 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8078f5-e2ea-42e5-b55b-018ce944dc9b admin admin] PATH_INFO: `/v3/projects/6b5668c24aef41a88140eafe6f35d447/users/2235e3832a014368b5298eaa9432f70d/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:32.552111 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8078f5-e2ea-42e5-b55b-018ce944dc9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b5668c24aef41a88140eafe6f35d447, user_id=2235e3832a014368b5298eaa9432f70d, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:32.554788 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8078f5-e2ea-42e5-b55b-018ce944dc9b admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:32.617795 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1327/2653] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:23:32 2022] PUT /identity/v3/projects/6b5668c24aef41a88140eafe6f35d447/users/2235e3832a014368b5298eaa9432f70d/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:23:32.621675 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7e1b4c-fe56-4fe0-b2a0-924fa7e22de6 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:32.631829 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7e1b4c-fe56-4fe0-b2a0-924fa7e22de6 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:32.652089 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:23:32.688452 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89acf31-e2d4-4940-9a21-3c3a1ebe0903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:32.689052 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89acf31-e2d4-4940-9a21-3c3a1ebe0903 admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:32.689326 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89acf31-e2d4-4940-9a21-3c3a1ebe0903 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:32.689591 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89acf31-e2d4-4940-9a21-3c3a1ebe0903 admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:32.690214 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89acf31-e2d4-4940-9a21-3c3a1ebe0903 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:32.691800 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89acf31-e2d4-4940-9a21-3c3a1ebe0903 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:32.701949 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1327/2654] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:23:32 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:23:32.706074 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8078f5-e2ea-42e5-b55b-018ce944dc9b admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:32.711540 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:23:32.717513 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8078f5-e2ea-42e5-b55b-018ce944dc9b admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:32.722540 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2151a62-f41c-4656-a266-2b73f9486740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:32.723179 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2151a62-f41c-4656-a266-2b73f9486740 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:32.723456 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2151a62-f41c-4656-a266-2b73f9486740 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:32.723727 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2151a62-f41c-4656-a266-2b73f9486740 admin admin] PATH_INFO: `/v3/projects/6b5668c24aef41a88140eafe6f35d447/users/2235e3832a014368b5298eaa9432f70d/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:32.725684 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2151a62-f41c-4656-a266-2b73f9486740 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b5668c24aef41a88140eafe6f35d447, user_id=2235e3832a014368b5298eaa9432f70d, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:32.728314 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2151a62-f41c-4656-a266-2b73f9486740 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:32.738486 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1328/2655] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:23:32 2022] PUT /identity/v3/projects/6b5668c24aef41a88140eafe6f35d447/users/2235e3832a014368b5298eaa9432f70d/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:23:32.748487 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed8f318-3331-492b-87f9-0dd82a74008d None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:32.751693 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed8f318-3331-492b-87f9-0dd82a74008d None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:32.751693 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed8f318-3331-492b-87f9-0dd82a74008d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:32.776342 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-9ed8f318-3331-492b-87f9-0dd82a74008d None None] MFA Rules not processed for user `2235e3832a014368b5298eaa9432f70d`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:23:32.777561 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-9ed8f318-3331-492b-87f9-0dd82a74008d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:23:32.845280 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1328/2656] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:23:32 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:23:32.857053 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2151a62-f41c-4656-a266-2b73f9486740 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:32.858827 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-b2151a62-f41c-4656-a266-2b73f9486740 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:23:32.916017 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2151a62-f41c-4656-a266-2b73f9486740 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:32.919652 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfe1bba-d803-488f-8e01-ebb7642fb437 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa71c71de263478da234f3c5716a6e5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:32.920646 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfe1bba-d803-488f-8e01-ebb7642fb437 service cinder] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:32.920935 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfe1bba-d803-488f-8e01-ebb7642fb437 service cinder] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:32.921206 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfe1bba-d803-488f-8e01-ebb7642fb437 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:32.930000 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfe1bba-d803-488f-8e01-ebb7642fb437 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:32.931412 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfe1bba-d803-488f-8e01-ebb7642fb437 service cinder] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:32.941137 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1329/2657] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:23:32 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:23:32.969977 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed8f318-3331-492b-87f9-0dd82a74008d None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:33.035278 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed8f318-3331-492b-87f9-0dd82a74008d None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:33.039777 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cab4750-19d8-4851-b45e-a0af2496511c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:33.040396 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cab4750-19d8-4851-b45e-a0af2496511c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:33.040702 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cab4750-19d8-4851-b45e-a0af2496511c admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:33.040975 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cab4750-19d8-4851-b45e-a0af2496511c admin admin] PATH_INFO: `/v3/users/ab179bc5d26c45dca8ad141c73469154` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:33.042111 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cab4750-19d8-4851-b45e-a0af2496511c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab179bc5d26c45dca8ad141c73469154)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:33.044021 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cab4750-19d8-4851-b45e-a0af2496511c admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:33.277245 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-3cab4750-19d8-4851-b45e-a0af2496511c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab179bc5d26c45dca8ad141c73469154'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:33.278336 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:23:33.290128 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-3cab4750-19d8-4851-b45e-a0af2496511c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab179bc5d26c45dca8ad141c73469154'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:33.371834 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-3cab4750-19d8-4851-b45e-a0af2496511c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab179bc5d26c45dca8ad141c73469154'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:33.379338 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-3cab4750-19d8-4851-b45e-a0af2496511c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab179bc5d26c45dca8ad141c73469154'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:33.381460 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-3cab4750-19d8-4851-b45e-a0af2496511c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab179bc5d26c45dca8ad141c73469154'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:33.386566 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1329/2658] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:23:32 2022] DELETE /identity/v3/users/ab179bc5d26c45dca8ad141c73469154 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:23:33.391736 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57232ecf-5dfb-426b-a622-a04402ad7a87 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:33.392020 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57232ecf-5dfb-426b-a622-a04402ad7a87 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:33.392293 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57232ecf-5dfb-426b-a622-a04402ad7a87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:33.411300 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-57232ecf-5dfb-426b-a622-a04402ad7a87 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:23:33.412518 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-57232ecf-5dfb-426b-a622-a04402ad7a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:23:33.469429 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1330/2659] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:23:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:23:33.479474 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cab4750-19d8-4851-b45e-a0af2496511c admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:33.480916 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-3cab4750-19d8-4851-b45e-a0af2496511c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:23:33.537884 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cab4750-19d8-4851-b45e-a0af2496511c admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:33.541564 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf3e02e-eeba-4fa8-b3fe-ae36bd335879 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:33.542102 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf3e02e-eeba-4fa8-b3fe-ae36bd335879 service neutron] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:33.542382 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf3e02e-eeba-4fa8-b3fe-ae36bd335879 service neutron] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:33.542661 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf3e02e-eeba-4fa8-b3fe-ae36bd335879 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:33.552329 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf3e02e-eeba-4fa8-b3fe-ae36bd335879 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:33.553779 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf3e02e-eeba-4fa8-b3fe-ae36bd335879 service neutron] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:33.563411 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1330/2660] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:23:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:23:33.985735 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57232ecf-5dfb-426b-a622-a04402ad7a87 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:33.987169 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-57232ecf-5dfb-426b-a622-a04402ad7a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:23:34.002340 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57232ecf-5dfb-426b-a622-a04402ad7a87 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:34.007095 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aeaf91-c931-400d-a8c2-63a2ad06dc68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:34.007732 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aeaf91-c931-400d-a8c2-63a2ad06dc68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:34.008010 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aeaf91-c931-400d-a8c2-63a2ad06dc68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:34.008282 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aeaf91-c931-400d-a8c2-63a2ad06dc68 admin admin] PATH_INFO: `/v3/projects/cffcd8aff30f4525bc2aa298f8c0de6f` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:34.009460 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aeaf91-c931-400d-a8c2-63a2ad06dc68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cffcd8aff30f4525bc2aa298f8c0de6f)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:34.011378 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aeaf91-c931-400d-a8c2-63a2ad06dc68 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:34.115377 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-29aeaf91-c931-400d-a8c2-63a2ad06dc68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cffcd8aff30f4525bc2aa298f8c0de6f', 'request_id': 'req-29aeaf91-c931-400d-a8c2-63a2ad06dc68'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:34.206553 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-29aeaf91-c931-400d-a8c2-63a2ad06dc68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cffcd8aff30f4525bc2aa298f8c0de6f', 'request_id': 'req-29aeaf91-c931-400d-a8c2-63a2ad06dc68'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:34.212566 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-29aeaf91-c931-400d-a8c2-63a2ad06dc68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cffcd8aff30f4525bc2aa298f8c0de6f', 'request_id': 'req-29aeaf91-c931-400d-a8c2-63a2ad06dc68'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:34.223928 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-29aeaf91-c931-400d-a8c2-63a2ad06dc68 admin admin] The token cache is being invalidate because project cffcd8aff30f4525bc2aa298f8c0de6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:23:34.224222 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-29aeaf91-c931-400d-a8c2-63a2ad06dc68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:34.224932 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-29aeaf91-c931-400d-a8c2-63a2ad06dc68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:34.226405 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1331/2661] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:23:33 2022] DELETE /identity/v3/projects/cffcd8aff30f4525bc2aa298f8c0de6f => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:23:34.230397 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf3e02e-eeba-4fa8-b3fe-ae36bd335879 service neutron] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:34.240921 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf3e02e-eeba-4fa8-b3fe-ae36bd335879 service neutron] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:34.299296 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cd31b5-7af3-4052-b07f-c055d79831f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:34.299986 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cd31b5-7af3-4052-b07f-c055d79831f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:34.300183 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cd31b5-7af3-4052-b07f-c055d79831f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:34.300493 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cd31b5-7af3-4052-b07f-c055d79831f3 admin admin] PATH_INFO: `/v3/users/2235e3832a014368b5298eaa9432f70d` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:34.301615 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cd31b5-7af3-4052-b07f-c055d79831f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2235e3832a014368b5298eaa9432f70d)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:34.303555 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cd31b5-7af3-4052-b07f-c055d79831f3 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:34.474134 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-a5cd31b5-7af3-4052-b07f-c055d79831f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2235e3832a014368b5298eaa9432f70d'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:34.486086 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-a5cd31b5-7af3-4052-b07f-c055d79831f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2235e3832a014368b5298eaa9432f70d'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:34.489638 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:23:34.557872 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-a5cd31b5-7af3-4052-b07f-c055d79831f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2235e3832a014368b5298eaa9432f70d'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:34.565766 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-a5cd31b5-7af3-4052-b07f-c055d79831f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2235e3832a014368b5298eaa9432f70d'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:34.566448 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-a5cd31b5-7af3-4052-b07f-c055d79831f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2235e3832a014368b5298eaa9432f70d'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:34.572444 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1331/2662] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:23:34 2022] DELETE /identity/v3/users/2235e3832a014368b5298eaa9432f70d => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:23:35.077943 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aeaf91-c931-400d-a8c2-63a2ad06dc68 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:35.077943 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-29aeaf91-c931-400d-a8c2-63a2ad06dc68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:23:35.091208 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aeaf91-c931-400d-a8c2-63a2ad06dc68 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:35.104996 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:23:35.149915 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb5cb1a-66ad-447f-8235-557748140bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:35.150536 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb5cb1a-66ad-447f-8235-557748140bae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:35.150842 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb5cb1a-66ad-447f-8235-557748140bae admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:35.151113 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb5cb1a-66ad-447f-8235-557748140bae admin admin] PATH_INFO: `/v3/projects/6b5668c24aef41a88140eafe6f35d447` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:35.152214 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb5cb1a-66ad-447f-8235-557748140bae admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b5668c24aef41a88140eafe6f35d447)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:35.154172 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb5cb1a-66ad-447f-8235-557748140bae admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:35.266110 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-fbb5cb1a-66ad-447f-8235-557748140bae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b5668c24aef41a88140eafe6f35d447', 'request_id': 'req-fbb5cb1a-66ad-447f-8235-557748140bae'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:35.366203 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-fbb5cb1a-66ad-447f-8235-557748140bae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b5668c24aef41a88140eafe6f35d447', 'request_id': 'req-fbb5cb1a-66ad-447f-8235-557748140bae'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:35.372234 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-fbb5cb1a-66ad-447f-8235-557748140bae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b5668c24aef41a88140eafe6f35d447', 'request_id': 'req-fbb5cb1a-66ad-447f-8235-557748140bae'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:35.383965 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-fbb5cb1a-66ad-447f-8235-557748140bae admin admin] The token cache is being invalidate because project 6b5668c24aef41a88140eafe6f35d447 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:23:35.384287 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-fbb5cb1a-66ad-447f-8235-557748140bae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:35.384976 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-fbb5cb1a-66ad-447f-8235-557748140bae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:35.386444 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1332/2663] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:23:35 2022] DELETE /identity/v3/projects/6b5668c24aef41a88140eafe6f35d447 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:23:35.391396 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4b86b0-5bde-41e9-98e0-a7a6ea822a60 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:35.391859 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4b86b0-5bde-41e9-98e0-a7a6ea822a60 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:35.391973 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4b86b0-5bde-41e9-98e0-a7a6ea822a60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:35.412135 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-3e4b86b0-5bde-41e9-98e0-a7a6ea822a60 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:23:35.413364 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-3e4b86b0-5bde-41e9-98e0-a7a6ea822a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:23:35.472778 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1332/2664] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:23:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:23:35.481572 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ff920d-86b3-4de2-94b2-887e27b5ba52 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:35.481849 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ff920d-86b3-4de2-94b2-887e27b5ba52 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:35.482119 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ff920d-86b3-4de2-94b2-887e27b5ba52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:35.501919 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-62ff920d-86b3-4de2-94b2-887e27b5ba52 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:23:35.503103 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-62ff920d-86b3-4de2-94b2-887e27b5ba52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:23:35.510768 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1333/2665] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:23:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:23:35.515707 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4b86b0-5bde-41e9-98e0-a7a6ea822a60 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:35.525513 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4b86b0-5bde-41e9-98e0-a7a6ea822a60 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:35.530769 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e2e74e-8fb5-4a97-a5b9-7ec28f0f899a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:35.531385 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e2e74e-8fb5-4a97-a5b9-7ec28f0f899a admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:35.531658 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e2e74e-8fb5-4a97-a5b9-7ec28f0f899a admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:35.531922 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e2e74e-8fb5-4a97-a5b9-7ec28f0f899a admin admin] PATH_INFO: `/v3/domains` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:35.532599 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e2e74e-8fb5-4a97-a5b9-7ec28f0f899a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 22:23:35.532893 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e2e74e-8fb5-4a97-a5b9-7ec28f0f899a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:35.534472 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e2e74e-8fb5-4a97-a5b9-7ec28f0f899a admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:35.546185 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1333/2666] 10.222.0.18 () {58 vars in 1153 bytes} [Sat Aug 6 22:23:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 22:23:35.550452 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ff920d-86b3-4de2-94b2-887e27b5ba52 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:35.559546 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ff920d-86b3-4de2-94b2-887e27b5ba52 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:35.564618 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f1531b-d673-4398-91aa-3f64e599c247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:35.565208 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f1531b-d673-4398-91aa-3f64e599c247 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:35.565487 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f1531b-d673-4398-91aa-3f64e599c247 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:35.565758 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f1531b-d673-4398-91aa-3f64e599c247 admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:35.566409 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f1531b-d673-4398-91aa-3f64e599c247 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:35.568298 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f1531b-d673-4398-91aa-3f64e599c247 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:35.647685 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1334/2667] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:23:35 2022] POST /identity/v3/projects => generated 379 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:23:35.651812 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e2e74e-8fb5-4a97-a5b9-7ec28f0f899a admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:35.662889 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e2e74e-8fb5-4a97-a5b9-7ec28f0f899a admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:35.718669 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e180c8d-56b6-4546-8c44-b7e0f3deea93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:35.719287 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e180c8d-56b6-4546-8c44-b7e0f3deea93 admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:35.719569 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e180c8d-56b6-4546-8c44-b7e0f3deea93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:35.719843 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e180c8d-56b6-4546-8c44-b7e0f3deea93 admin admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:35.720488 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e180c8d-56b6-4546-8c44-b7e0f3deea93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:35.722650 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e180c8d-56b6-4546-8c44-b7e0f3deea93 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:35.812021 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1334/2668] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:23:35 2022] POST /identity/v3/users => generated 290 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:23:35.816171 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f1531b-d673-4398-91aa-3f64e599c247 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:35.826465 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f1531b-d673-4398-91aa-3f64e599c247 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:35.831825 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b4456c-130a-4474-843c-c017da709577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:35.832429 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b4456c-130a-4474-843c-c017da709577 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:35.832726 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b4456c-130a-4474-843c-c017da709577 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:35.833000 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b4456c-130a-4474-843c-c017da709577 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:35.833638 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b4456c-130a-4474-843c-c017da709577 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:35.835227 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b4456c-130a-4474-843c-c017da709577 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:35.845821 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1335/2669] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:23:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:23:35.849925 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e180c8d-56b6-4546-8c44-b7e0f3deea93 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:35.860422 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e180c8d-56b6-4546-8c44-b7e0f3deea93 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:35.865113 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e8f131-2a61-4a33-8c5a-6809f9c9fbe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:35.865705 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e8f131-2a61-4a33-8c5a-6809f9c9fbe3 admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:35.865991 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e8f131-2a61-4a33-8c5a-6809f9c9fbe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:35.866288 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e8f131-2a61-4a33-8c5a-6809f9c9fbe3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:35.866927 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e8f131-2a61-4a33-8c5a-6809f9c9fbe3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:35.868596 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e8f131-2a61-4a33-8c5a-6809f9c9fbe3 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:35.880098 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1335/2670] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:23:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:23:35.884022 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b4456c-130a-4474-843c-c017da709577 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:35.894859 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b4456c-130a-4474-843c-c017da709577 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:35.899772 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21213fc2-9429-402d-8869-1101549adb16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:35.900352 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21213fc2-9429-402d-8869-1101549adb16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:35.900659 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21213fc2-9429-402d-8869-1101549adb16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:35.900932 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21213fc2-9429-402d-8869-1101549adb16 admin admin] PATH_INFO: `/v3/projects/4201abb8bdf041adbbe4d4e0bdb2e969/users/407f396d4eac4644895f4f195e11b870/roles/56e4a96094e9434cbbafb22579c034f5` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:35.915615 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21213fc2-9429-402d-8869-1101549adb16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4201abb8bdf041adbbe4d4e0bdb2e969, user_id=407f396d4eac4644895f4f195e11b870, role_id=56e4a96094e9434cbbafb22579c034f5)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:35.918309 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21213fc2-9429-402d-8869-1101549adb16 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:36.016287 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1336/2671] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:23:35 2022] PUT /identity/v3/projects/4201abb8bdf041adbbe4d4e0bdb2e969/users/407f396d4eac4644895f4f195e11b870/roles/56e4a96094e9434cbbafb22579c034f5 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:23:36.026391 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f2f415-2d6b-473b-ad07-1aff63eb22e3 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:36.026671 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f2f415-2d6b-473b-ad07-1aff63eb22e3 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:36.026940 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f2f415-2d6b-473b-ad07-1aff63eb22e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:36.053978 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-78f2f415-2d6b-473b-ad07-1aff63eb22e3 None None] MFA Rules not processed for user `407f396d4eac4644895f4f195e11b870`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:23:36.055168 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-78f2f415-2d6b-473b-ad07-1aff63eb22e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:23:36.097006 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:23:36.118807 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1336/2672] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:23:36 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:23:36.123551 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21213fc2-9429-402d-8869-1101549adb16 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:36.167638 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:23:36.183714 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21213fc2-9429-402d-8869-1101549adb16 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:36.189295 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f1ca8a-53a5-45e2-a480-8a1815e47029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:36.189895 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f1ca8a-53a5-45e2-a480-8a1815e47029 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:36.190171 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f1ca8a-53a5-45e2-a480-8a1815e47029 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:36.190439 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f1ca8a-53a5-45e2-a480-8a1815e47029 admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:36.191065 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f1ca8a-53a5-45e2-a480-8a1815e47029 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:36.193015 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f1ca8a-53a5-45e2-a480-8a1815e47029 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:36.290407 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1337/2673] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:23:36 2022] POST /identity/v3/projects => generated 381 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:23:36.294434 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f2f415-2d6b-473b-ad07-1aff63eb22e3 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:36.306387 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f2f415-2d6b-473b-ad07-1aff63eb22e3 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:36.363309 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d3277a-6549-4074-8ae2-3ed7045cc045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:36.364050 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d3277a-6549-4074-8ae2-3ed7045cc045 admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:36.364337 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d3277a-6549-4074-8ae2-3ed7045cc045 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:36.364637 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d3277a-6549-4074-8ae2-3ed7045cc045 admin admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:36.365302 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d3277a-6549-4074-8ae2-3ed7045cc045 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:36.367206 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d3277a-6549-4074-8ae2-3ed7045cc045 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:36.469322 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1337/2674] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:23:36 2022] POST /identity/v3/users => generated 297 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:23:36.473496 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f1ca8a-53a5-45e2-a480-8a1815e47029 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:36.483718 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f1ca8a-53a5-45e2-a480-8a1815e47029 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:36.488925 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18267df4-19ff-4910-be60-b3c2d815942b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:36.489510 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18267df4-19ff-4910-be60-b3c2d815942b admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:36.489787 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18267df4-19ff-4910-be60-b3c2d815942b admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:36.490058 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18267df4-19ff-4910-be60-b3c2d815942b admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:36.490694 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18267df4-19ff-4910-be60-b3c2d815942b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:36.492277 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18267df4-19ff-4910-be60-b3c2d815942b admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:36.502557 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1338/2675] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:23:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:23:36.506437 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d3277a-6549-4074-8ae2-3ed7045cc045 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:36.516335 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d3277a-6549-4074-8ae2-3ed7045cc045 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:36.520999 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac563e5-f79b-4f0f-895b-020eae0267f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:36.521641 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac563e5-f79b-4f0f-895b-020eae0267f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:36.521924 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac563e5-f79b-4f0f-895b-020eae0267f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:36.522200 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac563e5-f79b-4f0f-895b-020eae0267f3 admin admin] PATH_INFO: `/v3/projects/6875d39f1c4343deb790d0ab66f1bf3a/users/eb02ee06046748e8a66fc4baadab872f/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:36.536666 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac563e5-f79b-4f0f-895b-020eae0267f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6875d39f1c4343deb790d0ab66f1bf3a, user_id=eb02ee06046748e8a66fc4baadab872f, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:36.539294 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac563e5-f79b-4f0f-895b-020eae0267f3 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:36.601603 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1338/2676] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:23:36 2022] PUT /identity/v3/projects/6875d39f1c4343deb790d0ab66f1bf3a/users/eb02ee06046748e8a66fc4baadab872f/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:23:36.605573 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18267df4-19ff-4910-be60-b3c2d815942b admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:36.616152 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18267df4-19ff-4910-be60-b3c2d815942b admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:36.670478 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e5c996-2c6c-449e-9a4b-03f82b5f169f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:36.671073 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e5c996-2c6c-449e-9a4b-03f82b5f169f admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:36.672000 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e5c996-2c6c-449e-9a4b-03f82b5f169f admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:36.672255 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e5c996-2c6c-449e-9a4b-03f82b5f169f admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:36.672904 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e5c996-2c6c-449e-9a4b-03f82b5f169f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:36.674495 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e5c996-2c6c-449e-9a4b-03f82b5f169f admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:36.683906 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1339/2677] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:23:36 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:23:36.688088 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac563e5-f79b-4f0f-895b-020eae0267f3 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:36.698437 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac563e5-f79b-4f0f-895b-020eae0267f3 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:36.703130 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7e87bd-1c7a-418a-8106-17160f9d9df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:36.703799 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e87bd-1c7a-418a-8106-17160f9d9df8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:36.704074 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e87bd-1c7a-418a-8106-17160f9d9df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:36.704373 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e87bd-1c7a-418a-8106-17160f9d9df8 admin admin] PATH_INFO: `/v3/projects/6875d39f1c4343deb790d0ab66f1bf3a/users/eb02ee06046748e8a66fc4baadab872f/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:36.706313 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7e87bd-1c7a-418a-8106-17160f9d9df8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6875d39f1c4343deb790d0ab66f1bf3a, user_id=eb02ee06046748e8a66fc4baadab872f, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:36.708996 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7e87bd-1c7a-418a-8106-17160f9d9df8 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:36.719975 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1339/2678] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:23:36 2022] PUT /identity/v3/projects/6875d39f1c4343deb790d0ab66f1bf3a/users/eb02ee06046748e8a66fc4baadab872f/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:23:36.729939 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878fcb67-c25d-428c-8b64-9ae9d171935e None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:36.730223 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878fcb67-c25d-428c-8b64-9ae9d171935e None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:36.730496 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878fcb67-c25d-428c-8b64-9ae9d171935e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:36.757502 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-878fcb67-c25d-428c-8b64-9ae9d171935e None None] MFA Rules not processed for user `eb02ee06046748e8a66fc4baadab872f`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:23:36.758704 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-878fcb67-c25d-428c-8b64-9ae9d171935e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:23:36.826469 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1340/2679] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:23:36 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:23:36.838707 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7e87bd-1c7a-418a-8106-17160f9d9df8 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:36.840384 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-be7e87bd-1c7a-418a-8106-17160f9d9df8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:23:36.897918 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7e87bd-1c7a-418a-8106-17160f9d9df8 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:36.901639 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23986061-3364-4a12-8f80-5b157e42f0f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa71c71de263478da234f3c5716a6e5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:36.902239 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23986061-3364-4a12-8f80-5b157e42f0f8 service cinder] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:36.902520 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23986061-3364-4a12-8f80-5b157e42f0f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:36.902794 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23986061-3364-4a12-8f80-5b157e42f0f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:36.912366 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23986061-3364-4a12-8f80-5b157e42f0f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:36.913812 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23986061-3364-4a12-8f80-5b157e42f0f8 service cinder] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:36.923740 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1340/2680] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:23:36 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 22:23:37.902701 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878fcb67-c25d-428c-8b64-9ae9d171935e None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:37.912069 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878fcb67-c25d-428c-8b64-9ae9d171935e None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:37.916644 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7143f6-61bf-4e6b-a88e-2e5d94edb7ec tempest-VolumeQuotaClassesTest-1179230029 tempest-VolumeQuotaClassesTest-1179230029-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb02ee06046748e8a66fc4baadab872f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6875d39f1c4343deb790d0ab66f1bf3a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:37.917246 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7143f6-61bf-4e6b-a88e-2e5d94edb7ec tempest-VolumeQuotaClassesTest-1179230029 tempest-VolumeQuotaClassesTest-1179230029-project-admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:37.917563 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7143f6-61bf-4e6b-a88e-2e5d94edb7ec tempest-VolumeQuotaClassesTest-1179230029 tempest-VolumeQuotaClassesTest-1179230029-project-admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:37.917838 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7143f6-61bf-4e6b-a88e-2e5d94edb7ec tempest-VolumeQuotaClassesTest-1179230029 tempest-VolumeQuotaClassesTest-1179230029-project-admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:37.918480 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7143f6-61bf-4e6b-a88e-2e5d94edb7ec tempest-VolumeQuotaClassesTest-1179230029 tempest-VolumeQuotaClassesTest-1179230029-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 22:23:37.918781 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7143f6-61bf-4e6b-a88e-2e5d94edb7ec tempest-VolumeQuotaClassesTest-1179230029 tempest-VolumeQuotaClassesTest-1179230029-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:37.920365 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7143f6-61bf-4e6b-a88e-2e5d94edb7ec tempest-VolumeQuotaClassesTest-1179230029 tempest-VolumeQuotaClassesTest-1179230029-project-admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:37.932522 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1341/2681] 10.222.0.18 () {58 vars in 1153 bytes} [Sat Aug 6 22:23:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 22:23:37.936636 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23986061-3364-4a12-8f80-5b157e42f0f8 service cinder] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:37.946634 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23986061-3364-4a12-8f80-5b157e42f0f8 service cinder] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:37.951170 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbb3da2-7e2c-4698-973c-32835785a6d5 tempest-VolumeQuotaClassesTest-1179230029 tempest-VolumeQuotaClassesTest-1179230029-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb02ee06046748e8a66fc4baadab872f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6875d39f1c4343deb790d0ab66f1bf3a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:37.951757 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbb3da2-7e2c-4698-973c-32835785a6d5 tempest-VolumeQuotaClassesTest-1179230029 tempest-VolumeQuotaClassesTest-1179230029-project-admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:37.952034 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbb3da2-7e2c-4698-973c-32835785a6d5 tempest-VolumeQuotaClassesTest-1179230029 tempest-VolumeQuotaClassesTest-1179230029-project-admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:37.952304 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbb3da2-7e2c-4698-973c-32835785a6d5 tempest-VolumeQuotaClassesTest-1179230029 tempest-VolumeQuotaClassesTest-1179230029-project-admin] PATH_INFO: `/v3/projects` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:37.953171 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbb3da2-7e2c-4698-973c-32835785a6d5 tempest-VolumeQuotaClassesTest-1179230029 tempest-VolumeQuotaClassesTest-1179230029-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:37.955076 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbb3da2-7e2c-4698-973c-32835785a6d5 tempest-VolumeQuotaClassesTest-1179230029 tempest-VolumeQuotaClassesTest-1179230029-project-admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:38.065043 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1341/2682] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:23:37 2022] POST /identity/v3/projects => generated 354 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:23:38.069111 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7143f6-61bf-4e6b-a88e-2e5d94edb7ec tempest-VolumeQuotaClassesTest-1179230029 tempest-VolumeQuotaClassesTest-1179230029-project-admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:38.079180 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7143f6-61bf-4e6b-a88e-2e5d94edb7ec tempest-VolumeQuotaClassesTest-1179230029 tempest-VolumeQuotaClassesTest-1179230029-project-admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:38.100653 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:23:38.135508 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd639e4-0823-4b32-9489-73c654e1d96d tempest-VolumeQuotaClassesTest-1179230029 tempest-VolumeQuotaClassesTest-1179230029-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb02ee06046748e8a66fc4baadab872f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6875d39f1c4343deb790d0ab66f1bf3a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:38.136106 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd639e4-0823-4b32-9489-73c654e1d96d tempest-VolumeQuotaClassesTest-1179230029 tempest-VolumeQuotaClassesTest-1179230029-project-admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:38.136457 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd639e4-0823-4b32-9489-73c654e1d96d tempest-VolumeQuotaClassesTest-1179230029 tempest-VolumeQuotaClassesTest-1179230029-project-admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:38.142152 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd639e4-0823-4b32-9489-73c654e1d96d tempest-VolumeQuotaClassesTest-1179230029 tempest-VolumeQuotaClassesTest-1179230029-project-admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:38.142152 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd639e4-0823-4b32-9489-73c654e1d96d tempest-VolumeQuotaClassesTest-1179230029 tempest-VolumeQuotaClassesTest-1179230029-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 22:23:38.142152 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd639e4-0823-4b32-9489-73c654e1d96d tempest-VolumeQuotaClassesTest-1179230029 tempest-VolumeQuotaClassesTest-1179230029-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:38.142152 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd639e4-0823-4b32-9489-73c654e1d96d tempest-VolumeQuotaClassesTest-1179230029 tempest-VolumeQuotaClassesTest-1179230029-project-admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:38.151522 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1342/2683] 10.222.0.18 () {58 vars in 1153 bytes} [Sat Aug 6 22:23:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 22:23:38.190915 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbb3da2-7e2c-4698-973c-32835785a6d5 tempest-VolumeQuotaClassesTest-1179230029 tempest-VolumeQuotaClassesTest-1179230029-project-admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:38.201369 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbb3da2-7e2c-4698-973c-32835785a6d5 tempest-VolumeQuotaClassesTest-1179230029 tempest-VolumeQuotaClassesTest-1179230029-project-admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:38.206122 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db54f158-a3e9-496b-acfd-aa0254b01057 tempest-VolumeQuotaClassesTest-1179230029 tempest-VolumeQuotaClassesTest-1179230029-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb02ee06046748e8a66fc4baadab872f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6875d39f1c4343deb790d0ab66f1bf3a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:38.206748 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db54f158-a3e9-496b-acfd-aa0254b01057 tempest-VolumeQuotaClassesTest-1179230029 tempest-VolumeQuotaClassesTest-1179230029-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:38.207034 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db54f158-a3e9-496b-acfd-aa0254b01057 tempest-VolumeQuotaClassesTest-1179230029 tempest-VolumeQuotaClassesTest-1179230029-project-admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:38.207308 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db54f158-a3e9-496b-acfd-aa0254b01057 tempest-VolumeQuotaClassesTest-1179230029 tempest-VolumeQuotaClassesTest-1179230029-project-admin] PATH_INFO: `/v3/projects/657a947b8e0b44f28041fdc5250e46d9` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:38.208520 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db54f158-a3e9-496b-acfd-aa0254b01057 tempest-VolumeQuotaClassesTest-1179230029 tempest-VolumeQuotaClassesTest-1179230029-project-admin] RBAC: Authorizing `identity:delete_project(project_id=657a947b8e0b44f28041fdc5250e46d9)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:38.210414 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db54f158-a3e9-496b-acfd-aa0254b01057 tempest-VolumeQuotaClassesTest-1179230029 tempest-VolumeQuotaClassesTest-1179230029-project-admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:38.223766 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:23:38.326117 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-db54f158-a3e9-496b-acfd-aa0254b01057 tempest-VolumeQuotaClassesTest-1179230029 tempest-VolumeQuotaClassesTest-1179230029-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '657a947b8e0b44f28041fdc5250e46d9', 'request_id': 'req-db54f158-a3e9-496b-acfd-aa0254b01057'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:38.381291 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-db54f158-a3e9-496b-acfd-aa0254b01057 tempest-VolumeQuotaClassesTest-1179230029 tempest-VolumeQuotaClassesTest-1179230029-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '657a947b8e0b44f28041fdc5250e46d9', 'request_id': 'req-db54f158-a3e9-496b-acfd-aa0254b01057'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:38.387565 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-db54f158-a3e9-496b-acfd-aa0254b01057 tempest-VolumeQuotaClassesTest-1179230029 tempest-VolumeQuotaClassesTest-1179230029-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '657a947b8e0b44f28041fdc5250e46d9', 'request_id': 'req-db54f158-a3e9-496b-acfd-aa0254b01057'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:38.399424 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-db54f158-a3e9-496b-acfd-aa0254b01057 tempest-VolumeQuotaClassesTest-1179230029 tempest-VolumeQuotaClassesTest-1179230029-project-admin] The token cache is being invalidate because project 657a947b8e0b44f28041fdc5250e46d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:23:38.399709 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-db54f158-a3e9-496b-acfd-aa0254b01057 tempest-VolumeQuotaClassesTest-1179230029 tempest-VolumeQuotaClassesTest-1179230029-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:38.400371 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-db54f158-a3e9-496b-acfd-aa0254b01057 tempest-VolumeQuotaClassesTest-1179230029 tempest-VolumeQuotaClassesTest-1179230029-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:38.401940 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1342/2684] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:23:38 2022] DELETE /identity/v3/projects/657a947b8e0b44f28041fdc5250e46d9 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:23:39.001731 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd639e4-0823-4b32-9489-73c654e1d96d tempest-VolumeQuotaClassesTest-1179230029 tempest-VolumeQuotaClassesTest-1179230029-project-admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:39.003468 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-2cd639e4-0823-4b32-9489-73c654e1d96d tempest-VolumeQuotaClassesTest-1179230029 tempest-VolumeQuotaClassesTest-1179230029-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:23:39.067452 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd639e4-0823-4b32-9489-73c654e1d96d tempest-VolumeQuotaClassesTest-1179230029 tempest-VolumeQuotaClassesTest-1179230029-project-admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:39.121905 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e949a9d-1838-4e8a-845c-a1df510bc3e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:39.122521 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e949a9d-1838-4e8a-845c-a1df510bc3e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:39.122796 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e949a9d-1838-4e8a-845c-a1df510bc3e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:39.123067 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e949a9d-1838-4e8a-845c-a1df510bc3e3 admin admin] PATH_INFO: `/v3/users/407f396d4eac4644895f4f195e11b870` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:39.124178 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e949a9d-1838-4e8a-845c-a1df510bc3e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=407f396d4eac4644895f4f195e11b870)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:39.126748 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e949a9d-1838-4e8a-845c-a1df510bc3e3 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:39.266790 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-5e949a9d-1838-4e8a-845c-a1df510bc3e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '407f396d4eac4644895f4f195e11b870'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:39.279730 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-5e949a9d-1838-4e8a-845c-a1df510bc3e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '407f396d4eac4644895f4f195e11b870'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:39.281962 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:23:39.287720 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-5e949a9d-1838-4e8a-845c-a1df510bc3e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '407f396d4eac4644895f4f195e11b870'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:39.288607 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-5e949a9d-1838-4e8a-845c-a1df510bc3e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '407f396d4eac4644895f4f195e11b870'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:39.289210 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-5e949a9d-1838-4e8a-845c-a1df510bc3e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '407f396d4eac4644895f4f195e11b870'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:39.365026 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1343/2685] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:23:39 2022] DELETE /identity/v3/users/407f396d4eac4644895f4f195e11b870 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:23:39.369472 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c317548-c4c6-4130-9148-febf84b8ada8 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:39.369948 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c317548-c4c6-4130-9148-febf84b8ada8 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:39.370133 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c317548-c4c6-4130-9148-febf84b8ada8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:39.389166 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-4c317548-c4c6-4130-9148-febf84b8ada8 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:23:39.390357 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-4c317548-c4c6-4130-9148-febf84b8ada8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:23:39.449960 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1343/2686] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:23:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:23:39.459596 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e949a9d-1838-4e8a-845c-a1df510bc3e3 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:39.461037 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-5e949a9d-1838-4e8a-845c-a1df510bc3e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:23:39.517419 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e949a9d-1838-4e8a-845c-a1df510bc3e3 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:39.520953 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6f3562-33de-43e9-90b5-9528420ecb0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:39.521546 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6f3562-33de-43e9-90b5-9528420ecb0f service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:39.521820 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6f3562-33de-43e9-90b5-9528420ecb0f service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:39.522087 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6f3562-33de-43e9-90b5-9528420ecb0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:39.531330 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6f3562-33de-43e9-90b5-9528420ecb0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:39.532780 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6f3562-33de-43e9-90b5-9528420ecb0f service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:39.541974 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1344/2687] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:23:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:23:39.914399 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c317548-c4c6-4130-9148-febf84b8ada8 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:39.915979 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-4c317548-c4c6-4130-9148-febf84b8ada8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:23:39.931657 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c317548-c4c6-4130-9148-febf84b8ada8 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:39.937298 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cc31c3-6d32-4743-9afc-23fbc989c409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:39.937984 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc31c3-6d32-4743-9afc-23fbc989c409 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:39.938275 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc31c3-6d32-4743-9afc-23fbc989c409 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:39.938716 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc31c3-6d32-4743-9afc-23fbc989c409 admin admin] PATH_INFO: `/v3/projects/4201abb8bdf041adbbe4d4e0bdb2e969` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:39.939904 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cc31c3-6d32-4743-9afc-23fbc989c409 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4201abb8bdf041adbbe4d4e0bdb2e969)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:39.941965 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cc31c3-6d32-4743-9afc-23fbc989c409 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:40.090594 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-73cc31c3-6d32-4743-9afc-23fbc989c409 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4201abb8bdf041adbbe4d4e0bdb2e969', 'request_id': 'req-73cc31c3-6d32-4743-9afc-23fbc989c409'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:40.211768 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-73cc31c3-6d32-4743-9afc-23fbc989c409 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4201abb8bdf041adbbe4d4e0bdb2e969', 'request_id': 'req-73cc31c3-6d32-4743-9afc-23fbc989c409'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:40.217945 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-73cc31c3-6d32-4743-9afc-23fbc989c409 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4201abb8bdf041adbbe4d4e0bdb2e969', 'request_id': 'req-73cc31c3-6d32-4743-9afc-23fbc989c409'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:40.230025 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-73cc31c3-6d32-4743-9afc-23fbc989c409 admin admin] The token cache is being invalidate because project 4201abb8bdf041adbbe4d4e0bdb2e969 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:23:40.230305 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-73cc31c3-6d32-4743-9afc-23fbc989c409 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:40.230922 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-73cc31c3-6d32-4743-9afc-23fbc989c409 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:40.232396 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1344/2688] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:23:39 2022] DELETE /identity/v3/projects/4201abb8bdf041adbbe4d4e0bdb2e969 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:23:40.236582 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6f3562-33de-43e9-90b5-9528420ecb0f service neutron] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:40.246179 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6f3562-33de-43e9-90b5-9528420ecb0f service neutron] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:40.304622 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db46b02-750c-4365-b006-2d321e15cac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:40.305253 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db46b02-750c-4365-b006-2d321e15cac2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:40.305532 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db46b02-750c-4365-b006-2d321e15cac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:40.305827 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db46b02-750c-4365-b006-2d321e15cac2 admin admin] PATH_INFO: `/v3/users/eb02ee06046748e8a66fc4baadab872f` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:40.307020 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db46b02-750c-4365-b006-2d321e15cac2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb02ee06046748e8a66fc4baadab872f)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:40.308999 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db46b02-750c-4365-b006-2d321e15cac2 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:40.493818 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-3db46b02-750c-4365-b006-2d321e15cac2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb02ee06046748e8a66fc4baadab872f'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:40.506706 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-3db46b02-750c-4365-b006-2d321e15cac2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb02ee06046748e8a66fc4baadab872f'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:40.513695 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-3db46b02-750c-4365-b006-2d321e15cac2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb02ee06046748e8a66fc4baadab872f'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:40.514402 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-3db46b02-750c-4365-b006-2d321e15cac2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb02ee06046748e8a66fc4baadab872f'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:40.515598 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-3db46b02-750c-4365-b006-2d321e15cac2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb02ee06046748e8a66fc4baadab872f'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:40.604356 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1345/2689] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:23:40 2022] DELETE /identity/v3/users/eb02ee06046748e8a66fc4baadab872f => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:23:41.016907 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cc31c3-6d32-4743-9afc-23fbc989c409 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:41.020088 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-73cc31c3-6d32-4743-9afc-23fbc989c409 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:23:41.031710 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cc31c3-6d32-4743-9afc-23fbc989c409 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:41.050546 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:23:41.088123 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4e83a7-c1bf-4242-8d13-61213dd7892e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:41.088803 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e83a7-c1bf-4242-8d13-61213dd7892e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:41.089082 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e83a7-c1bf-4242-8d13-61213dd7892e admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:41.089356 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e83a7-c1bf-4242-8d13-61213dd7892e admin admin] PATH_INFO: `/v3/projects/6875d39f1c4343deb790d0ab66f1bf3a` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:41.090530 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4e83a7-c1bf-4242-8d13-61213dd7892e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6875d39f1c4343deb790d0ab66f1bf3a)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:41.092493 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4e83a7-c1bf-4242-8d13-61213dd7892e admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:41.194350 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-cf4e83a7-c1bf-4242-8d13-61213dd7892e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6875d39f1c4343deb790d0ab66f1bf3a', 'request_id': 'req-cf4e83a7-c1bf-4242-8d13-61213dd7892e'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:41.262988 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-cf4e83a7-c1bf-4242-8d13-61213dd7892e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6875d39f1c4343deb790d0ab66f1bf3a', 'request_id': 'req-cf4e83a7-c1bf-4242-8d13-61213dd7892e'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:41.268518 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-cf4e83a7-c1bf-4242-8d13-61213dd7892e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6875d39f1c4343deb790d0ab66f1bf3a', 'request_id': 'req-cf4e83a7-c1bf-4242-8d13-61213dd7892e'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:41.279895 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-cf4e83a7-c1bf-4242-8d13-61213dd7892e admin admin] The token cache is being invalidate because project 6875d39f1c4343deb790d0ab66f1bf3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:23:41.280180 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-cf4e83a7-c1bf-4242-8d13-61213dd7892e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:41.281464 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-cf4e83a7-c1bf-4242-8d13-61213dd7892e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:41.284209 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1345/2690] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:23:41 2022] DELETE /identity/v3/projects/6875d39f1c4343deb790d0ab66f1bf3a => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:23:41.287848 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf3e25f-e24f-422d-ac26-067043c21c0f None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:41.288330 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf3e25f-e24f-422d-ac26-067043c21c0f None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:41.288471 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf3e25f-e24f-422d-ac26-067043c21c0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:41.302406 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:23:41.308362 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-1cf3e25f-e24f-422d-ac26-067043c21c0f None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:23:41.309585 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-1cf3e25f-e24f-422d-ac26-067043c21c0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:23:41.368255 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1346/2691] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:23:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:23:41.376343 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe96ce1-37f5-4c68-af47-19a0295f68e0 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:41.376669 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe96ce1-37f5-4c68-af47-19a0295f68e0 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:41.376944 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe96ce1-37f5-4c68-af47-19a0295f68e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:41.396141 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-0fe96ce1-37f5-4c68-af47-19a0295f68e0 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:23:41.397372 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-0fe96ce1-37f5-4c68-af47-19a0295f68e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:23:41.406054 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1346/2692] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:23:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:23:41.410553 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf3e25f-e24f-422d-ac26-067043c21c0f None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:41.419939 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf3e25f-e24f-422d-ac26-067043c21c0f None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:41.425146 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4934314e-4bd8-483a-bce2-0fdc4d078a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:41.425734 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4934314e-4bd8-483a-bce2-0fdc4d078a6c admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:41.426015 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4934314e-4bd8-483a-bce2-0fdc4d078a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:41.426283 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4934314e-4bd8-483a-bce2-0fdc4d078a6c admin admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:41.426921 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4934314e-4bd8-483a-bce2-0fdc4d078a6c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 22:23:41.427224 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4934314e-4bd8-483a-bce2-0fdc4d078a6c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:41.428869 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4934314e-4bd8-483a-bce2-0fdc4d078a6c admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:41.440082 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1347/2693] 10.222.0.18 () {58 vars in 1153 bytes} [Sat Aug 6 22:23:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 22:23:41.444240 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe96ce1-37f5-4c68-af47-19a0295f68e0 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:41.454217 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe96ce1-37f5-4c68-af47-19a0295f68e0 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:41.458872 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f1bf0f-3e43-4914-8797-2bfaa70ca87d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:41.459483 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f1bf0f-3e43-4914-8797-2bfaa70ca87d admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:41.459765 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f1bf0f-3e43-4914-8797-2bfaa70ca87d admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:41.460041 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f1bf0f-3e43-4914-8797-2bfaa70ca87d admin admin] PATH_INFO: `/v3/projects` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:41.460709 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f1bf0f-3e43-4914-8797-2bfaa70ca87d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:41.462613 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f1bf0f-3e43-4914-8797-2bfaa70ca87d admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:41.555966 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1347/2694] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:23:41 2022] POST /identity/v3/projects => generated 385 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:23:41.565487 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4934314e-4bd8-483a-bce2-0fdc4d078a6c admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:41.575641 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4934314e-4bd8-483a-bce2-0fdc4d078a6c admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:41.631479 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f8e9c5-ab12-40a6-be3c-f1bb0bf56118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:41.632074 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f8e9c5-ab12-40a6-be3c-f1bb0bf56118 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:41.632353 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f8e9c5-ab12-40a6-be3c-f1bb0bf56118 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:41.632656 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f8e9c5-ab12-40a6-be3c-f1bb0bf56118 admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:41.633266 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f8e9c5-ab12-40a6-be3c-f1bb0bf56118 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:41.635169 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f8e9c5-ab12-40a6-be3c-f1bb0bf56118 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:41.721448 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1348/2695] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:23:41 2022] POST /identity/v3/users => generated 293 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 22:23:41.725829 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f1bf0f-3e43-4914-8797-2bfaa70ca87d admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:41.736005 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f1bf0f-3e43-4914-8797-2bfaa70ca87d admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:41.740624 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa27c550-3508-4f24-ad8b-a4155da9c25d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:41.741223 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27c550-3508-4f24-ad8b-a4155da9c25d admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:41.741505 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27c550-3508-4f24-ad8b-a4155da9c25d admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:41.741779 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27c550-3508-4f24-ad8b-a4155da9c25d admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:41.742435 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa27c550-3508-4f24-ad8b-a4155da9c25d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:41.743997 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa27c550-3508-4f24-ad8b-a4155da9c25d admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:41.753806 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1348/2696] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:23:41 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:23:41.757601 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f8e9c5-ab12-40a6-be3c-f1bb0bf56118 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:41.768110 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f8e9c5-ab12-40a6-be3c-f1bb0bf56118 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:41.772690 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38516398-efe4-43e9-ba64-bdb80e6e27ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:41.773273 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38516398-efe4-43e9-ba64-bdb80e6e27ff admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:41.773553 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38516398-efe4-43e9-ba64-bdb80e6e27ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:41.773854 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38516398-efe4-43e9-ba64-bdb80e6e27ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:41.774519 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38516398-efe4-43e9-ba64-bdb80e6e27ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:41.776097 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38516398-efe4-43e9-ba64-bdb80e6e27ff admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:41.788389 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1349/2697] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:23:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:23:41.792425 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa27c550-3508-4f24-ad8b-a4155da9c25d admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:41.802751 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa27c550-3508-4f24-ad8b-a4155da9c25d admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:41.816125 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708bef3d-7afe-4416-abbe-18b6a2e1532a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:41.816125 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708bef3d-7afe-4416-abbe-18b6a2e1532a admin admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:41.816125 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708bef3d-7afe-4416-abbe-18b6a2e1532a admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:41.816125 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708bef3d-7afe-4416-abbe-18b6a2e1532a admin admin] PATH_INFO: `/v3/projects/fe515b9ee88d47d4880d5fdbb619a6b5/users/4db546e83a774ac99136c5b4bd636249/roles/56e4a96094e9434cbbafb22579c034f5` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:41.823872 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708bef3d-7afe-4416-abbe-18b6a2e1532a admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe515b9ee88d47d4880d5fdbb619a6b5, user_id=4db546e83a774ac99136c5b4bd636249, role_id=56e4a96094e9434cbbafb22579c034f5)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:41.826528 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708bef3d-7afe-4416-abbe-18b6a2e1532a admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:41.920492 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1349/2698] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:23:41 2022] PUT /identity/v3/projects/fe515b9ee88d47d4880d5fdbb619a6b5/users/4db546e83a774ac99136c5b4bd636249/roles/56e4a96094e9434cbbafb22579c034f5 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:23:41.931006 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc54e149-5a19-4ceb-86ec-77964bd8f77f None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:41.931309 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc54e149-5a19-4ceb-86ec-77964bd8f77f None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:41.931585 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc54e149-5a19-4ceb-86ec-77964bd8f77f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:41.960094 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-bc54e149-5a19-4ceb-86ec-77964bd8f77f None None] MFA Rules not processed for user `4db546e83a774ac99136c5b4bd636249`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:23:41.961330 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-bc54e149-5a19-4ceb-86ec-77964bd8f77f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:23:42.025179 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1350/2699] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:23:41 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:23:42.036055 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708bef3d-7afe-4416-abbe-18b6a2e1532a admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:42.037761 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-708bef3d-7afe-4416-abbe-18b6a2e1532a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:23:42.092871 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:23:42.096964 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708bef3d-7afe-4416-abbe-18b6a2e1532a admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:42.100648 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba6465f-f4fe-4f48-81b4-26478a4812a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa71c71de263478da234f3c5716a6e5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:42.101246 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba6465f-f4fe-4f48-81b4-26478a4812a0 service cinder] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:42.101524 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba6465f-f4fe-4f48-81b4-26478a4812a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:42.113764 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba6465f-f4fe-4f48-81b4-26478a4812a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:42.113764 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba6465f-f4fe-4f48-81b4-26478a4812a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:42.113764 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba6465f-f4fe-4f48-81b4-26478a4812a0 service cinder] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:42.119892 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1350/2700] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:23:42 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:23:42.138937 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc54e149-5a19-4ceb-86ec-77964bd8f77f None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:42.144744 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:23:42.199922 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc54e149-5a19-4ceb-86ec-77964bd8f77f None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:42.205431 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6e16df-a44d-42a4-b662-bcf9d139f84a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:42.206062 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e16df-a44d-42a4-b662-bcf9d139f84a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:42.206343 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e16df-a44d-42a4-b662-bcf9d139f84a admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:42.206620 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e16df-a44d-42a4-b662-bcf9d139f84a admin admin] PATH_INFO: `/v3/users/4db546e83a774ac99136c5b4bd636249` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:42.207775 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6e16df-a44d-42a4-b662-bcf9d139f84a admin admin] RBAC: Authorizing `identity:delete_user(user_id=4db546e83a774ac99136c5b4bd636249)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:42.209805 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6e16df-a44d-42a4-b662-bcf9d139f84a admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:42.418121 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-fa6e16df-a44d-42a4-b662-bcf9d139f84a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4db546e83a774ac99136c5b4bd636249'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:42.430115 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-fa6e16df-a44d-42a4-b662-bcf9d139f84a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4db546e83a774ac99136c5b4bd636249'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:42.437108 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-fa6e16df-a44d-42a4-b662-bcf9d139f84a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4db546e83a774ac99136c5b4bd636249'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:42.437863 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-fa6e16df-a44d-42a4-b662-bcf9d139f84a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4db546e83a774ac99136c5b4bd636249'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:42.438498 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-fa6e16df-a44d-42a4-b662-bcf9d139f84a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4db546e83a774ac99136c5b4bd636249'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:42.520311 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1351/2701] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:23:42 2022] DELETE /identity/v3/users/4db546e83a774ac99136c5b4bd636249 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:23:42.524802 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92194b-229d-471f-bf1d-bbc076ef3f83 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:42.525083 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92194b-229d-471f-bf1d-bbc076ef3f83 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:42.525353 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92194b-229d-471f-bf1d-bbc076ef3f83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:42.545475 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-fd92194b-229d-471f-bf1d-bbc076ef3f83 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:23:42.546690 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-fd92194b-229d-471f-bf1d-bbc076ef3f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:23:42.604450 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1351/2702] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:23:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:23:42.614306 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6e16df-a44d-42a4-b662-bcf9d139f84a admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:42.615759 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-fa6e16df-a44d-42a4-b662-bcf9d139f84a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:23:42.672031 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6e16df-a44d-42a4-b662-bcf9d139f84a admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:42.675599 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58af90f3-5207-4799-8846-564506093331 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:42.676224 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58af90f3-5207-4799-8846-564506093331 service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:42.676543 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58af90f3-5207-4799-8846-564506093331 service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:42.676817 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58af90f3-5207-4799-8846-564506093331 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:42.685645 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58af90f3-5207-4799-8846-564506093331 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:42.687682 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58af90f3-5207-4799-8846-564506093331 service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:42.697186 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1352/2703] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:23:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:23:43.091017 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd92194b-229d-471f-bf1d-bbc076ef3f83 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:43.092577 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-fd92194b-229d-471f-bf1d-bbc076ef3f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:23:43.105274 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd92194b-229d-471f-bf1d-bbc076ef3f83 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:43.110816 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6633d2-ed45-4ce4-98bd-6759e74fc88d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:43.111457 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6633d2-ed45-4ce4-98bd-6759e74fc88d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:43.111752 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6633d2-ed45-4ce4-98bd-6759e74fc88d admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:43.112158 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6633d2-ed45-4ce4-98bd-6759e74fc88d admin admin] PATH_INFO: `/v3/projects/fe515b9ee88d47d4880d5fdbb619a6b5` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:43.113345 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6633d2-ed45-4ce4-98bd-6759e74fc88d admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe515b9ee88d47d4880d5fdbb619a6b5)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:43.115436 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6633d2-ed45-4ce4-98bd-6759e74fc88d admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:43.205621 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-9c6633d2-ed45-4ce4-98bd-6759e74fc88d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe515b9ee88d47d4880d5fdbb619a6b5', 'request_id': 'req-9c6633d2-ed45-4ce4-98bd-6759e74fc88d'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:43.263100 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-9c6633d2-ed45-4ce4-98bd-6759e74fc88d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe515b9ee88d47d4880d5fdbb619a6b5', 'request_id': 'req-9c6633d2-ed45-4ce4-98bd-6759e74fc88d'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:43.269029 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-9c6633d2-ed45-4ce4-98bd-6759e74fc88d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe515b9ee88d47d4880d5fdbb619a6b5', 'request_id': 'req-9c6633d2-ed45-4ce4-98bd-6759e74fc88d'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:43.280129 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-9c6633d2-ed45-4ce4-98bd-6759e74fc88d admin admin] The token cache is being invalidate because project fe515b9ee88d47d4880d5fdbb619a6b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:23:43.280557 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-9c6633d2-ed45-4ce4-98bd-6759e74fc88d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:43.281207 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-9c6633d2-ed45-4ce4-98bd-6759e74fc88d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:43.282678 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1352/2704] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:23:43 2022] DELETE /identity/v3/projects/fe515b9ee88d47d4880d5fdbb619a6b5 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:23:43.287458 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad45ebb-632b-4c3c-a55a-df89285d6df5 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:43.287745 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad45ebb-632b-4c3c-a55a-df89285d6df5 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:43.287999 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad45ebb-632b-4c3c-a55a-df89285d6df5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:43.307684 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-cad45ebb-632b-4c3c-a55a-df89285d6df5 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:23:43.308939 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-cad45ebb-632b-4c3c-a55a-df89285d6df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:23:43.327724 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:23:43.369461 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1353/2705] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:23:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 22:23:43.378599 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dc1029-2387-4a83-bdb4-d1b787e3493c None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:43.378866 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dc1029-2387-4a83-bdb4-d1b787e3493c None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:43.379140 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dc1029-2387-4a83-bdb4-d1b787e3493c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:43.398673 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-c8dc1029-2387-4a83-bdb4-d1b787e3493c None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:23:43.399966 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-c8dc1029-2387-4a83-bdb4-d1b787e3493c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:23:43.408911 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1353/2706] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:23:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:23:43.413157 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad45ebb-632b-4c3c-a55a-df89285d6df5 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:43.424506 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad45ebb-632b-4c3c-a55a-df89285d6df5 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:43.429159 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9de494-46c9-4cc9-9d95-12700d07ebe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:43.429749 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9de494-46c9-4cc9-9d95-12700d07ebe7 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:43.430025 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9de494-46c9-4cc9-9d95-12700d07ebe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:43.430294 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9de494-46c9-4cc9-9d95-12700d07ebe7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:43.430924 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9de494-46c9-4cc9-9d95-12700d07ebe7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 22:23:43.431223 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9de494-46c9-4cc9-9d95-12700d07ebe7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:43.432875 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9de494-46c9-4cc9-9d95-12700d07ebe7 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:43.444248 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1354/2707] 10.222.0.18 () {58 vars in 1153 bytes} [Sat Aug 6 22:23:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 22:23:43.448437 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dc1029-2387-4a83-bdb4-d1b787e3493c None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:43.457865 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dc1029-2387-4a83-bdb4-d1b787e3493c None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:43.462431 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108c139f-ab39-4daa-bc16-209d2081a64f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:43.463019 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108c139f-ab39-4daa-bc16-209d2081a64f admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:43.463299 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108c139f-ab39-4daa-bc16-209d2081a64f admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:43.463570 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108c139f-ab39-4daa-bc16-209d2081a64f admin admin] PATH_INFO: `/v3/projects` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:43.464199 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108c139f-ab39-4daa-bc16-209d2081a64f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:43.466152 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108c139f-ab39-4daa-bc16-209d2081a64f admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:43.585220 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1354/2708] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:23:43 2022] POST /identity/v3/projects => generated 385 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:23:43.589481 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9de494-46c9-4cc9-9d95-12700d07ebe7 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:43.600275 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9de494-46c9-4cc9-9d95-12700d07ebe7 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:43.655115 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cf37db-aa0f-4fc5-9c7e-5cafeed41bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:43.656311 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cf37db-aa0f-4fc5-9c7e-5cafeed41bd6 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:43.656640 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cf37db-aa0f-4fc5-9c7e-5cafeed41bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:43.656915 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cf37db-aa0f-4fc5-9c7e-5cafeed41bd6 admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:43.657527 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cf37db-aa0f-4fc5-9c7e-5cafeed41bd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:43.659440 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cf37db-aa0f-4fc5-9c7e-5cafeed41bd6 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:43.759847 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1355/2709] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:23:43 2022] POST /identity/v3/users => generated 293 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:23:43.764086 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108c139f-ab39-4daa-bc16-209d2081a64f admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:43.775119 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108c139f-ab39-4daa-bc16-209d2081a64f admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:43.779910 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196960f0-d5ed-4eb9-a768-bde7dd2e46b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:43.781200 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196960f0-d5ed-4eb9-a768-bde7dd2e46b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:43.781481 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196960f0-d5ed-4eb9-a768-bde7dd2e46b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:43.781751 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196960f0-d5ed-4eb9-a768-bde7dd2e46b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:43.782391 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196960f0-d5ed-4eb9-a768-bde7dd2e46b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:43.783998 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196960f0-d5ed-4eb9-a768-bde7dd2e46b8 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:43.794209 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1355/2710] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:23:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:23:43.798250 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cf37db-aa0f-4fc5-9c7e-5cafeed41bd6 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:43.808118 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cf37db-aa0f-4fc5-9c7e-5cafeed41bd6 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:43.812768 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698a5401-96a9-4828-8fec-ca1724bf7935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:43.813353 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698a5401-96a9-4828-8fec-ca1724bf7935 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:43.813633 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698a5401-96a9-4828-8fec-ca1724bf7935 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:43.813903 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698a5401-96a9-4828-8fec-ca1724bf7935 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:43.814534 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698a5401-96a9-4828-8fec-ca1724bf7935 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:43.816146 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698a5401-96a9-4828-8fec-ca1724bf7935 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:43.826521 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1356/2711] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:23:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:23:43.831180 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196960f0-d5ed-4eb9-a768-bde7dd2e46b8 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:43.841392 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196960f0-d5ed-4eb9-a768-bde7dd2e46b8 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:43.847051 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3122dd-8253-4f17-86fe-32c1156261bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:43.847686 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3122dd-8253-4f17-86fe-32c1156261bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:43.847958 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3122dd-8253-4f17-86fe-32c1156261bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:43.848228 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3122dd-8253-4f17-86fe-32c1156261bc admin admin] PATH_INFO: `/v3/projects/03fe57467d954b2b952f78a6690bf243/users/e91e80c08b554e09a7cdc70e273ae9a1/roles/56e4a96094e9434cbbafb22579c034f5` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:43.862544 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3122dd-8253-4f17-86fe-32c1156261bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=03fe57467d954b2b952f78a6690bf243, user_id=e91e80c08b554e09a7cdc70e273ae9a1, role_id=56e4a96094e9434cbbafb22579c034f5)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:43.865204 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3122dd-8253-4f17-86fe-32c1156261bc admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:43.943239 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1356/2712] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:23:43 2022] PUT /identity/v3/projects/03fe57467d954b2b952f78a6690bf243/users/e91e80c08b554e09a7cdc70e273ae9a1/roles/56e4a96094e9434cbbafb22579c034f5 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:23:44.007640 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17a55a5-0dd7-4e95-a828-2d995b6b3a63 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:44.007918 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17a55a5-0dd7-4e95-a828-2d995b6b3a63 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:44.008176 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17a55a5-0dd7-4e95-a828-2d995b6b3a63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:44.036336 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-c17a55a5-0dd7-4e95-a828-2d995b6b3a63 None None] MFA Rules not processed for user `e91e80c08b554e09a7cdc70e273ae9a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:23:44.037498 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-c17a55a5-0dd7-4e95-a828-2d995b6b3a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:23:44.099198 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1357/2713] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:23:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:23:44.103846 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3122dd-8253-4f17-86fe-32c1156261bc admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:44.116594 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:23:44.166678 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3122dd-8253-4f17-86fe-32c1156261bc admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:44.171856 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cda513-3262-44d7-bf4a-afa13c552f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:44.172686 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cda513-3262-44d7-bf4a-afa13c552f3c admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:44.173279 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cda513-3262-44d7-bf4a-afa13c552f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:44.173604 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cda513-3262-44d7-bf4a-afa13c552f3c admin admin] PATH_INFO: `/v3/projects` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:44.174234 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cda513-3262-44d7-bf4a-afa13c552f3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:44.176161 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cda513-3262-44d7-bf4a-afa13c552f3c admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:44.298687 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1357/2714] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:23:44 2022] POST /identity/v3/projects => generated 383 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:23:44.302744 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17a55a5-0dd7-4e95-a828-2d995b6b3a63 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:44.313894 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17a55a5-0dd7-4e95-a828-2d995b6b3a63 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:44.330343 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:23:44.370861 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616e85bc-f52a-4258-a878-4dd9d1765f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:44.371461 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616e85bc-f52a-4258-a878-4dd9d1765f73 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:44.371745 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616e85bc-f52a-4258-a878-4dd9d1765f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:44.372015 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616e85bc-f52a-4258-a878-4dd9d1765f73 admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:44.372672 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616e85bc-f52a-4258-a878-4dd9d1765f73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:44.375149 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616e85bc-f52a-4258-a878-4dd9d1765f73 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:44.466995 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1358/2715] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:23:44 2022] POST /identity/v3/users => generated 292 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:23:44.471026 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cda513-3262-44d7-bf4a-afa13c552f3c admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:44.484058 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cda513-3262-44d7-bf4a-afa13c552f3c admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:44.488791 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17b4c9c-1dd1-4ed3-b393-39ea693de0f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:44.489381 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b4c9c-1dd1-4ed3-b393-39ea693de0f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:44.489660 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b4c9c-1dd1-4ed3-b393-39ea693de0f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:44.489932 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b4c9c-1dd1-4ed3-b393-39ea693de0f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:44.490567 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17b4c9c-1dd1-4ed3-b393-39ea693de0f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:44.492152 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17b4c9c-1dd1-4ed3-b393-39ea693de0f9 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:44.503389 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1358/2716] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:23:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:23:44.507425 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616e85bc-f52a-4258-a878-4dd9d1765f73 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:44.517398 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616e85bc-f52a-4258-a878-4dd9d1765f73 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:44.521975 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bd03e4-029c-4dca-951b-39209d2f85f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:44.522553 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bd03e4-029c-4dca-951b-39209d2f85f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:44.522828 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bd03e4-029c-4dca-951b-39209d2f85f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:44.523094 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bd03e4-029c-4dca-951b-39209d2f85f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:44.523750 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bd03e4-029c-4dca-951b-39209d2f85f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:44.525353 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bd03e4-029c-4dca-951b-39209d2f85f4 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:44.534886 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1359/2717] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:23:44 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:23:44.538826 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17b4c9c-1dd1-4ed3-b393-39ea693de0f9 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:44.549416 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17b4c9c-1dd1-4ed3-b393-39ea693de0f9 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:44.554108 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957f4d54-318e-469d-91e9-8b76d20dd1df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:44.554750 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f4d54-318e-469d-91e9-8b76d20dd1df admin admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:44.555033 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f4d54-318e-469d-91e9-8b76d20dd1df admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:44.555307 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f4d54-318e-469d-91e9-8b76d20dd1df admin admin] PATH_INFO: `/v3/projects/c0ba0522e9064077b16b2b2a98a081ee/users/8ffc6329a4794cf58c00b02f36aa82d5/roles/56e4a96094e9434cbbafb22579c034f5` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:44.570632 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957f4d54-318e-469d-91e9-8b76d20dd1df admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0ba0522e9064077b16b2b2a98a081ee, user_id=8ffc6329a4794cf58c00b02f36aa82d5, role_id=56e4a96094e9434cbbafb22579c034f5)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:44.573340 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957f4d54-318e-469d-91e9-8b76d20dd1df admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:44.674666 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1359/2718] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:23:44 2022] PUT /identity/v3/projects/c0ba0522e9064077b16b2b2a98a081ee/users/8ffc6329a4794cf58c00b02f36aa82d5/roles/56e4a96094e9434cbbafb22579c034f5 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 22:23:44.684392 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecf8bd1-3104-4264-8501-68cf55fe38f5 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:44.684704 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecf8bd1-3104-4264-8501-68cf55fe38f5 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:44.684979 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecf8bd1-3104-4264-8501-68cf55fe38f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:44.712693 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-4ecf8bd1-3104-4264-8501-68cf55fe38f5 None None] MFA Rules not processed for user `8ffc6329a4794cf58c00b02f36aa82d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:23:44.713895 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-4ecf8bd1-3104-4264-8501-68cf55fe38f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:23:44.776827 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1360/2719] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:23:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:23:44.781459 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957f4d54-318e-469d-91e9-8b76d20dd1df admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:44.843211 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957f4d54-318e-469d-91e9-8b76d20dd1df admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:44.848818 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6642d7ec-4409-4e5e-978b-d847ebf7b98f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:44.849411 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6642d7ec-4409-4e5e-978b-d847ebf7b98f admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:44.849687 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6642d7ec-4409-4e5e-978b-d847ebf7b98f admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:44.849956 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6642d7ec-4409-4e5e-978b-d847ebf7b98f admin admin] PATH_INFO: `/v3/projects` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:44.850580 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6642d7ec-4409-4e5e-978b-d847ebf7b98f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:44.852538 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6642d7ec-4409-4e5e-978b-d847ebf7b98f admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:44.973823 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1360/2720] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:23:44 2022] POST /identity/v3/projects => generated 385 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:23:44.977544 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecf8bd1-3104-4264-8501-68cf55fe38f5 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:44.988540 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecf8bd1-3104-4264-8501-68cf55fe38f5 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:45.042381 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5406c66-df11-48da-bba0-533853364536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:45.042979 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5406c66-df11-48da-bba0-533853364536 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:45.043254 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5406c66-df11-48da-bba0-533853364536 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:45.043522 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5406c66-df11-48da-bba0-533853364536 admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:45.044141 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5406c66-df11-48da-bba0-533853364536 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:45.046073 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5406c66-df11-48da-bba0-533853364536 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:45.125721 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1361/2721] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:23:44 2022] POST /identity/v3/users => generated 299 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:23:45.129623 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6642d7ec-4409-4e5e-978b-d847ebf7b98f admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:45.139735 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6642d7ec-4409-4e5e-978b-d847ebf7b98f admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:45.144685 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5871bf7c-043d-4644-92a4-7870ce8a4fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:45.145273 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5871bf7c-043d-4644-92a4-7870ce8a4fd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:45.145551 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5871bf7c-043d-4644-92a4-7870ce8a4fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:45.145868 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5871bf7c-043d-4644-92a4-7870ce8a4fd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:45.146485 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5871bf7c-043d-4644-92a4-7870ce8a4fd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:45.148051 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5871bf7c-043d-4644-92a4-7870ce8a4fd8 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:45.159204 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1361/2722] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:23:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:23:45.167212 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5406c66-df11-48da-bba0-533853364536 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:45.177988 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5406c66-df11-48da-bba0-533853364536 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:45.182619 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b0aa41-1aa6-4846-ae30-844fcafbc711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:45.183254 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b0aa41-1aa6-4846-ae30-844fcafbc711 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:45.183531 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b0aa41-1aa6-4846-ae30-844fcafbc711 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:45.183803 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b0aa41-1aa6-4846-ae30-844fcafbc711 admin admin] PATH_INFO: `/v3/projects/8ef1bcdbc5ae48ecad4c5e6b9bc3d5ec/users/1e6f037fb315499f93cd2caddfa69976/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:45.198307 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b0aa41-1aa6-4846-ae30-844fcafbc711 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ef1bcdbc5ae48ecad4c5e6b9bc3d5ec, user_id=1e6f037fb315499f93cd2caddfa69976, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:45.200946 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b0aa41-1aa6-4846-ae30-844fcafbc711 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:45.203528 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:23:45.262877 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1362/2723] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:23:45 2022] PUT /identity/v3/projects/8ef1bcdbc5ae48ecad4c5e6b9bc3d5ec/users/1e6f037fb315499f93cd2caddfa69976/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:23:45.267244 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5871bf7c-043d-4644-92a4-7870ce8a4fd8 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:45.277331 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5871bf7c-043d-4644-92a4-7870ce8a4fd8 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:45.309616 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:23:45.333442 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c6c17f-7c48-48da-9b6a-3a7d2742a07d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:45.334033 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c6c17f-7c48-48da-9b6a-3a7d2742a07d admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:45.334312 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c6c17f-7c48-48da-9b6a-3a7d2742a07d admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:45.334591 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c6c17f-7c48-48da-9b6a-3a7d2742a07d admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:45.335234 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c6c17f-7c48-48da-9b6a-3a7d2742a07d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:45.336844 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c6c17f-7c48-48da-9b6a-3a7d2742a07d admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:45.347190 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1362/2724] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:23:45 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:23:45.351761 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b0aa41-1aa6-4846-ae30-844fcafbc711 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:45.362499 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b0aa41-1aa6-4846-ae30-844fcafbc711 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:45.367076 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20560fe1-dda0-4d3a-923d-86b7a1222c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:45.367729 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20560fe1-dda0-4d3a-923d-86b7a1222c46 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:45.374817 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20560fe1-dda0-4d3a-923d-86b7a1222c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:45.374817 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20560fe1-dda0-4d3a-923d-86b7a1222c46 admin admin] PATH_INFO: `/v3/projects/8ef1bcdbc5ae48ecad4c5e6b9bc3d5ec/users/1e6f037fb315499f93cd2caddfa69976/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:45.374817 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20560fe1-dda0-4d3a-923d-86b7a1222c46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ef1bcdbc5ae48ecad4c5e6b9bc3d5ec, user_id=1e6f037fb315499f93cd2caddfa69976, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:45.374817 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20560fe1-dda0-4d3a-923d-86b7a1222c46 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:45.383154 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1363/2725] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:23:45 2022] PUT /identity/v3/projects/8ef1bcdbc5ae48ecad4c5e6b9bc3d5ec/users/1e6f037fb315499f93cd2caddfa69976/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:23:45.395062 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6017f6-d7f0-4687-93d2-5574d9577344 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:45.395382 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6017f6-d7f0-4687-93d2-5574d9577344 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:45.395669 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6017f6-d7f0-4687-93d2-5574d9577344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:45.425240 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-8c6017f6-d7f0-4687-93d2-5574d9577344 None None] MFA Rules not processed for user `1e6f037fb315499f93cd2caddfa69976`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:23:45.426460 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-8c6017f6-d7f0-4687-93d2-5574d9577344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:23:45.495269 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1363/2726] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:23:45 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:23:45.506334 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20560fe1-dda0-4d3a-923d-86b7a1222c46 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:45.508103 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-20560fe1-dda0-4d3a-923d-86b7a1222c46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:23:45.566404 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20560fe1-dda0-4d3a-923d-86b7a1222c46 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:45.569954 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a07403-7ec6-42af-a8af-7140cfb265d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa71c71de263478da234f3c5716a6e5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:45.570544 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a07403-7ec6-42af-a8af-7140cfb265d7 service cinder] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:45.570817 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a07403-7ec6-42af-a8af-7140cfb265d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:45.571113 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a07403-7ec6-42af-a8af-7140cfb265d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:45.626274 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a07403-7ec6-42af-a8af-7140cfb265d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:45.627699 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a07403-7ec6-42af-a8af-7140cfb265d7 service cinder] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:45.636479 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1364/2727] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:23:45 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:23:47.523976 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6017f6-d7f0-4687-93d2-5574d9577344 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:47.528082 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-8c6017f6-d7f0-4687-93d2-5574d9577344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:23:47.538285 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6017f6-d7f0-4687-93d2-5574d9577344 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:47.541821 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf385ed-564a-4b38-bd25-9ffb44513ba6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa71c71de263478da234f3c5716a6e5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:47.542417 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf385ed-564a-4b38-bd25-9ffb44513ba6 service cinder] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:47.542697 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf385ed-564a-4b38-bd25-9ffb44513ba6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:47.542974 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf385ed-564a-4b38-bd25-9ffb44513ba6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:47.600888 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf385ed-564a-4b38-bd25-9ffb44513ba6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:47.602303 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf385ed-564a-4b38-bd25-9ffb44513ba6 service cinder] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:47.611092 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1364/2728] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:23:47 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:23:49.065680 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a07403-7ec6-42af-a8af-7140cfb265d7 service cinder] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:49.074163 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a07403-7ec6-42af-a8af-7140cfb265d7 service cinder] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:49.077726 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a87a1b-7a3d-44e5-bdba-6271e651856c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa71c71de263478da234f3c5716a6e5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:49.078310 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a87a1b-7a3d-44e5-bdba-6271e651856c service cinder] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:49.078584 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a87a1b-7a3d-44e5-bdba-6271e651856c service cinder] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:49.078851 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a87a1b-7a3d-44e5-bdba-6271e651856c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:49.088032 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a87a1b-7a3d-44e5-bdba-6271e651856c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:49.089465 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a87a1b-7a3d-44e5-bdba-6271e651856c service cinder] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:49.099457 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1365/2729] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:23:49 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 22:23:53.595990 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf385ed-564a-4b38-bd25-9ffb44513ba6 service cinder] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:53.653125 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=100 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:23:53.658614 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf385ed-564a-4b38-bd25-9ffb44513ba6 service cinder] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:53.662997 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e929517-fa69-4f3d-8c3a-83a66a078651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:53.667278 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e929517-fa69-4f3d-8c3a-83a66a078651 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:53.667278 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e929517-fa69-4f3d-8c3a-83a66a078651 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:53.667278 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e929517-fa69-4f3d-8c3a-83a66a078651 admin admin] PATH_INFO: `/v3/users/e91e80c08b554e09a7cdc70e273ae9a1` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:53.667278 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e929517-fa69-4f3d-8c3a-83a66a078651 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e91e80c08b554e09a7cdc70e273ae9a1)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:53.667278 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e929517-fa69-4f3d-8c3a-83a66a078651 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:53.879246 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-4e929517-fa69-4f3d-8c3a-83a66a078651 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e91e80c08b554e09a7cdc70e273ae9a1'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:53.890833 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-4e929517-fa69-4f3d-8c3a-83a66a078651 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e91e80c08b554e09a7cdc70e273ae9a1'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:53.977177 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-4e929517-fa69-4f3d-8c3a-83a66a078651 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e91e80c08b554e09a7cdc70e273ae9a1'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:53.984477 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-4e929517-fa69-4f3d-8c3a-83a66a078651 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e91e80c08b554e09a7cdc70e273ae9a1'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:53.985123 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-4e929517-fa69-4f3d-8c3a-83a66a078651 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e91e80c08b554e09a7cdc70e273ae9a1'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:53.991586 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1365/2730] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:23:53 2022] DELETE /identity/v3/users/e91e80c08b554e09a7cdc70e273ae9a1 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:23:53.996061 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e687acf8-ebfa-472c-9830-587c02a0f3e7 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:53.996589 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e687acf8-ebfa-472c-9830-587c02a0f3e7 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:53.996689 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e687acf8-ebfa-472c-9830-587c02a0f3e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:54.015414 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-e687acf8-ebfa-472c-9830-587c02a0f3e7 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:23:54.016630 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-e687acf8-ebfa-472c-9830-587c02a0f3e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:23:54.073778 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1366/2731] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:23:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:23:54.083883 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e929517-fa69-4f3d-8c3a-83a66a078651 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:54.085306 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-4e929517-fa69-4f3d-8c3a-83a66a078651 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:23:54.141789 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e929517-fa69-4f3d-8c3a-83a66a078651 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:54.145393 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754ed6b5-380c-4211-980b-7dbab2091b99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:54.145991 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754ed6b5-380c-4211-980b-7dbab2091b99 service neutron] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:54.146270 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754ed6b5-380c-4211-980b-7dbab2091b99 service neutron] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:54.146540 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754ed6b5-380c-4211-980b-7dbab2091b99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:54.155406 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754ed6b5-380c-4211-980b-7dbab2091b99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:54.156848 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754ed6b5-380c-4211-980b-7dbab2091b99 service neutron] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:54.166138 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1366/2732] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:23:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:23:54.572957 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e687acf8-ebfa-472c-9830-587c02a0f3e7 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:54.574363 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-e687acf8-ebfa-472c-9830-587c02a0f3e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:23:54.587769 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e687acf8-ebfa-472c-9830-587c02a0f3e7 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:54.592660 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854d83c2-7827-4f68-95a1-3395d2aa0b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:54.593296 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854d83c2-7827-4f68-95a1-3395d2aa0b99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:54.593575 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854d83c2-7827-4f68-95a1-3395d2aa0b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:54.593847 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854d83c2-7827-4f68-95a1-3395d2aa0b99 admin admin] PATH_INFO: `/v3/projects/03fe57467d954b2b952f78a6690bf243` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:54.595411 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854d83c2-7827-4f68-95a1-3395d2aa0b99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03fe57467d954b2b952f78a6690bf243)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:54.597964 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854d83c2-7827-4f68-95a1-3395d2aa0b99 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:54.615714 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:23:54.717320 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-854d83c2-7827-4f68-95a1-3395d2aa0b99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03fe57467d954b2b952f78a6690bf243', 'request_id': 'req-854d83c2-7827-4f68-95a1-3395d2aa0b99'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:54.780367 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-854d83c2-7827-4f68-95a1-3395d2aa0b99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03fe57467d954b2b952f78a6690bf243', 'request_id': 'req-854d83c2-7827-4f68-95a1-3395d2aa0b99'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:54.786985 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-854d83c2-7827-4f68-95a1-3395d2aa0b99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03fe57467d954b2b952f78a6690bf243', 'request_id': 'req-854d83c2-7827-4f68-95a1-3395d2aa0b99'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:54.798421 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-854d83c2-7827-4f68-95a1-3395d2aa0b99 admin admin] The token cache is being invalidate because project 03fe57467d954b2b952f78a6690bf243 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:23:54.799406 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-854d83c2-7827-4f68-95a1-3395d2aa0b99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:54.800073 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-854d83c2-7827-4f68-95a1-3395d2aa0b99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:54.801579 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1367/2733] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:23:54 2022] DELETE /identity/v3/projects/03fe57467d954b2b952f78a6690bf243 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:23:54.805531 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754ed6b5-380c-4211-980b-7dbab2091b99 service neutron] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:54.815539 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754ed6b5-380c-4211-980b-7dbab2091b99 service neutron] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:54.874388 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993d1d4c-5716-4bf6-bab2-68c33f78329b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:54.874934 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993d1d4c-5716-4bf6-bab2-68c33f78329b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:54.875242 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993d1d4c-5716-4bf6-bab2-68c33f78329b admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:54.875517 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993d1d4c-5716-4bf6-bab2-68c33f78329b admin admin] PATH_INFO: `/v3/users/8ffc6329a4794cf58c00b02f36aa82d5` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:54.876667 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993d1d4c-5716-4bf6-bab2-68c33f78329b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ffc6329a4794cf58c00b02f36aa82d5)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:54.878581 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993d1d4c-5716-4bf6-bab2-68c33f78329b admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:54.899084 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:23:55.030964 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-993d1d4c-5716-4bf6-bab2-68c33f78329b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ffc6329a4794cf58c00b02f36aa82d5'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:55.043945 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-993d1d4c-5716-4bf6-bab2-68c33f78329b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ffc6329a4794cf58c00b02f36aa82d5'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:55.102252 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-993d1d4c-5716-4bf6-bab2-68c33f78329b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ffc6329a4794cf58c00b02f36aa82d5'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:55.109531 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-993d1d4c-5716-4bf6-bab2-68c33f78329b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ffc6329a4794cf58c00b02f36aa82d5'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:55.110160 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-993d1d4c-5716-4bf6-bab2-68c33f78329b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ffc6329a4794cf58c00b02f36aa82d5'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:55.116240 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1367/2734] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:23:54 2022] DELETE /identity/v3/users/8ffc6329a4794cf58c00b02f36aa82d5 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:23:55.519698 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854d83c2-7827-4f68-95a1-3395d2aa0b99 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:55.521296 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-854d83c2-7827-4f68-95a1-3395d2aa0b99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:23:55.534923 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854d83c2-7827-4f68-95a1-3395d2aa0b99 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:55.592409 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71531a9-5285-4240-9922-f5013a259e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:55.593055 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71531a9-5285-4240-9922-f5013a259e98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:55.593335 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71531a9-5285-4240-9922-f5013a259e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:55.593613 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71531a9-5285-4240-9922-f5013a259e98 admin admin] PATH_INFO: `/v3/projects/c0ba0522e9064077b16b2b2a98a081ee` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:55.594722 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71531a9-5285-4240-9922-f5013a259e98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0ba0522e9064077b16b2b2a98a081ee)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:55.597329 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71531a9-5285-4240-9922-f5013a259e98 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:55.707989 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-a71531a9-5285-4240-9922-f5013a259e98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0ba0522e9064077b16b2b2a98a081ee', 'request_id': 'req-a71531a9-5285-4240-9922-f5013a259e98'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:55.777257 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-a71531a9-5285-4240-9922-f5013a259e98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0ba0522e9064077b16b2b2a98a081ee', 'request_id': 'req-a71531a9-5285-4240-9922-f5013a259e98'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:55.783310 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-a71531a9-5285-4240-9922-f5013a259e98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0ba0522e9064077b16b2b2a98a081ee', 'request_id': 'req-a71531a9-5285-4240-9922-f5013a259e98'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:55.794316 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-a71531a9-5285-4240-9922-f5013a259e98 admin admin] The token cache is being invalidate because project c0ba0522e9064077b16b2b2a98a081ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:23:55.802543 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-a71531a9-5285-4240-9922-f5013a259e98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:55.802543 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-a71531a9-5285-4240-9922-f5013a259e98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:55.802543 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1368/2735] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:23:55 2022] DELETE /identity/v3/projects/c0ba0522e9064077b16b2b2a98a081ee => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:23:55.802873 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993d1d4c-5716-4bf6-bab2-68c33f78329b admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:55.802873 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-993d1d4c-5716-4bf6-bab2-68c33f78329b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:23:55.816315 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993d1d4c-5716-4bf6-bab2-68c33f78329b admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:55.872829 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7ab529-04ba-4f71-96f9-b13e16e87383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:55.873455 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ab529-04ba-4f71-96f9-b13e16e87383 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:55.873733 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ab529-04ba-4f71-96f9-b13e16e87383 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:55.874006 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ab529-04ba-4f71-96f9-b13e16e87383 admin admin] PATH_INFO: `/v3/users/1e6f037fb315499f93cd2caddfa69976` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:55.875121 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7ab529-04ba-4f71-96f9-b13e16e87383 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e6f037fb315499f93cd2caddfa69976)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:55.877896 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7ab529-04ba-4f71-96f9-b13e16e87383 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:56.042256 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-fa7ab529-04ba-4f71-96f9-b13e16e87383 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e6f037fb315499f93cd2caddfa69976'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:56.054609 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-fa7ab529-04ba-4f71-96f9-b13e16e87383 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e6f037fb315499f93cd2caddfa69976'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:56.123330 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-fa7ab529-04ba-4f71-96f9-b13e16e87383 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e6f037fb315499f93cd2caddfa69976'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:56.130769 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-fa7ab529-04ba-4f71-96f9-b13e16e87383 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e6f037fb315499f93cd2caddfa69976'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:56.131465 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-fa7ab529-04ba-4f71-96f9-b13e16e87383 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e6f037fb315499f93cd2caddfa69976'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:56.137793 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1368/2736] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:23:55 2022] DELETE /identity/v3/users/1e6f037fb315499f93cd2caddfa69976 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:23:56.605194 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71531a9-5285-4240-9922-f5013a259e98 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:56.606650 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-a71531a9-5285-4240-9922-f5013a259e98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:23:56.621377 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71531a9-5285-4240-9922-f5013a259e98 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:56.654937 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:23:56.679197 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00471d2-b66c-4d85-bd47-b49e41288f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:56.679819 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00471d2-b66c-4d85-bd47-b49e41288f92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:56.680095 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00471d2-b66c-4d85-bd47-b49e41288f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:56.680363 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00471d2-b66c-4d85-bd47-b49e41288f92 admin admin] PATH_INFO: `/v3/projects/8ef1bcdbc5ae48ecad4c5e6b9bc3d5ec` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:56.681504 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00471d2-b66c-4d85-bd47-b49e41288f92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ef1bcdbc5ae48ecad4c5e6b9bc3d5ec)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:56.683397 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00471d2-b66c-4d85-bd47-b49e41288f92 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:56.777229 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-a00471d2-b66c-4d85-bd47-b49e41288f92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ef1bcdbc5ae48ecad4c5e6b9bc3d5ec', 'request_id': 'req-a00471d2-b66c-4d85-bd47-b49e41288f92'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:56.834314 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-a00471d2-b66c-4d85-bd47-b49e41288f92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ef1bcdbc5ae48ecad4c5e6b9bc3d5ec', 'request_id': 'req-a00471d2-b66c-4d85-bd47-b49e41288f92'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:56.840292 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-a00471d2-b66c-4d85-bd47-b49e41288f92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ef1bcdbc5ae48ecad4c5e6b9bc3d5ec', 'request_id': 'req-a00471d2-b66c-4d85-bd47-b49e41288f92'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:56.851723 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-a00471d2-b66c-4d85-bd47-b49e41288f92 admin admin] The token cache is being invalidate because project 8ef1bcdbc5ae48ecad4c5e6b9bc3d5ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:23:56.851945 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-a00471d2-b66c-4d85-bd47-b49e41288f92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:56.852610 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-a00471d2-b66c-4d85-bd47-b49e41288f92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:23:56.854107 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1369/2737] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:23:56 2022] DELETE /identity/v3/projects/8ef1bcdbc5ae48ecad4c5e6b9bc3d5ec => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:23:56.865728 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f644eef7-492c-4dbe-8ac2-9d79fc147edb None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:56.866234 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f644eef7-492c-4dbe-8ac2-9d79fc147edb None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:56.866335 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f644eef7-492c-4dbe-8ac2-9d79fc147edb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:56.887336 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-f644eef7-492c-4dbe-8ac2-9d79fc147edb None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:23:56.888622 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-f644eef7-492c-4dbe-8ac2-9d79fc147edb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:23:56.898972 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:23:56.952424 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1369/2738] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:23:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:23:56.961191 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5aace5-c967-451a-9451-0e4b0d124c8b None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:56.961498 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5aace5-c967-451a-9451-0e4b0d124c8b None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:56.961826 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5aace5-c967-451a-9451-0e4b0d124c8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:56.982262 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-dd5aace5-c967-451a-9451-0e4b0d124c8b None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:23:56.983469 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-dd5aace5-c967-451a-9451-0e4b0d124c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:23:56.991669 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1370/2739] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:23:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:23:57.003273 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f644eef7-492c-4dbe-8ac2-9d79fc147edb None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:57.013265 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f644eef7-492c-4dbe-8ac2-9d79fc147edb None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:57.020854 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb66f84-305f-4cd0-b479-325ce17b10db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:57.021797 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb66f84-305f-4cd0-b479-325ce17b10db admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:57.022259 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb66f84-305f-4cd0-b479-325ce17b10db admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:57.022701 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb66f84-305f-4cd0-b479-325ce17b10db admin admin] PATH_INFO: `/v3/domains` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:57.023805 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb66f84-305f-4cd0-b479-325ce17b10db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 22:23:57.024302 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb66f84-305f-4cd0-b479-325ce17b10db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:57.026785 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb66f84-305f-4cd0-b479-325ce17b10db admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:57.038599 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1370/2740] 10.222.0.18 () {58 vars in 1153 bytes} [Sat Aug 6 22:23:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 22:23:57.043310 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5aace5-c967-451a-9451-0e4b0d124c8b None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:57.053049 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5aace5-c967-451a-9451-0e4b0d124c8b None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:57.057646 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a695f1d4-ec44-4c8e-976a-e77c9bfb39b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:57.058236 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a695f1d4-ec44-4c8e-976a-e77c9bfb39b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:57.058517 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a695f1d4-ec44-4c8e-976a-e77c9bfb39b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:57.058785 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a695f1d4-ec44-4c8e-976a-e77c9bfb39b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:57.059408 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a695f1d4-ec44-4c8e-976a-e77c9bfb39b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:57.061677 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a695f1d4-ec44-4c8e-976a-e77c9bfb39b3 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:57.161589 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1371/2741] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:23:57 2022] POST /identity/v3/projects => generated 373 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:23:57.164500 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb66f84-305f-4cd0-b479-325ce17b10db admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:57.175416 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb66f84-305f-4cd0-b479-325ce17b10db admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:57.234763 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50ffaaa-f6e9-4362-8308-cc83e9c5ecaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:57.235368 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50ffaaa-f6e9-4362-8308-cc83e9c5ecaa admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:57.235646 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50ffaaa-f6e9-4362-8308-cc83e9c5ecaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:57.236442 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50ffaaa-f6e9-4362-8308-cc83e9c5ecaa admin admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:57.237159 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50ffaaa-f6e9-4362-8308-cc83e9c5ecaa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:57.239355 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50ffaaa-f6e9-4362-8308-cc83e9c5ecaa admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:57.318389 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1371/2742] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:23:57 2022] POST /identity/v3/users => generated 287 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:23:57.323071 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a695f1d4-ec44-4c8e-976a-e77c9bfb39b3 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:57.336273 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a695f1d4-ec44-4c8e-976a-e77c9bfb39b3 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:57.343421 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f97391-3c83-46a0-89da-ee568b2d1275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:57.344499 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f97391-3c83-46a0-89da-ee568b2d1275 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:57.344923 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f97391-3c83-46a0-89da-ee568b2d1275 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:57.345354 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f97391-3c83-46a0-89da-ee568b2d1275 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:57.346142 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f97391-3c83-46a0-89da-ee568b2d1275 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:57.348006 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f97391-3c83-46a0-89da-ee568b2d1275 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:57.360687 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1372/2743] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:23:57 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:23:57.365212 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50ffaaa-f6e9-4362-8308-cc83e9c5ecaa admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:57.377938 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50ffaaa-f6e9-4362-8308-cc83e9c5ecaa admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:57.382827 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8da6879-fc45-4073-bfbe-f9e407ed2638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:57.383424 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8da6879-fc45-4073-bfbe-f9e407ed2638 admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:57.383736 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8da6879-fc45-4073-bfbe-f9e407ed2638 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:57.384014 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8da6879-fc45-4073-bfbe-f9e407ed2638 admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:57.384687 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8da6879-fc45-4073-bfbe-f9e407ed2638 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:57.386375 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8da6879-fc45-4073-bfbe-f9e407ed2638 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:57.397671 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1372/2744] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:23:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:23:57.401843 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f97391-3c83-46a0-89da-ee568b2d1275 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:57.412304 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f97391-3c83-46a0-89da-ee568b2d1275 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:57.416827 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f251d085-96ab-4d00-93c2-c275f4664f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:57.417474 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f251d085-96ab-4d00-93c2-c275f4664f55 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:57.417776 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f251d085-96ab-4d00-93c2-c275f4664f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:57.418074 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f251d085-96ab-4d00-93c2-c275f4664f55 admin admin] PATH_INFO: `/v3/projects/676d7214383f4677989de5282569bd67/users/15d2106935444c97b1e8c50131d3171f/roles/56e4a96094e9434cbbafb22579c034f5` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:57.435973 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f251d085-96ab-4d00-93c2-c275f4664f55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=676d7214383f4677989de5282569bd67, user_id=15d2106935444c97b1e8c50131d3171f, role_id=56e4a96094e9434cbbafb22579c034f5)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:57.440853 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f251d085-96ab-4d00-93c2-c275f4664f55 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:57.544630 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1373/2745] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:23:57 2022] PUT /identity/v3/projects/676d7214383f4677989de5282569bd67/users/15d2106935444c97b1e8c50131d3171f/roles/56e4a96094e9434cbbafb22579c034f5 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:23:57.558511 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6927fb1-4c8e-4ea2-92a8-f4735a01b571 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:57.558804 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6927fb1-4c8e-4ea2-92a8-f4735a01b571 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:57.560485 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6927fb1-4c8e-4ea2-92a8-f4735a01b571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:57.588616 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-d6927fb1-4c8e-4ea2-92a8-f4735a01b571 None None] MFA Rules not processed for user `15d2106935444c97b1e8c50131d3171f`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:23:57.589866 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-d6927fb1-4c8e-4ea2-92a8-f4735a01b571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:23:57.658470 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1373/2746] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:23:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:23:57.670551 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f251d085-96ab-4d00-93c2-c275f4664f55 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:23:57.672303 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-f251d085-96ab-4d00-93c2-c275f4664f55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:23:57.744262 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f251d085-96ab-4d00-93c2-c275f4664f55 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:23:57.744262 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210ae65e-806b-4ac9-94ef-78af549c3310 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa71c71de263478da234f3c5716a6e5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:23:57.744262 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210ae65e-806b-4ac9-94ef-78af549c3310 service cinder] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:57.744262 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210ae65e-806b-4ac9-94ef-78af549c3310 service cinder] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:57.745132 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210ae65e-806b-4ac9-94ef-78af549c3310 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:57.754945 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210ae65e-806b-4ac9-94ef-78af549c3310 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:23:57.756574 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210ae65e-806b-4ac9-94ef-78af549c3310 service cinder] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:23:57.766186 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1374/2747] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:23:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:23:59.568712 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702641cd-38e5-4682-98e8-66319de27692 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:23:59.568712 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702641cd-38e5-4682-98e8-66319de27692 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:23:59.568712 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702641cd-38e5-4682-98e8-66319de27692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:23:59.583848 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-702641cd-38e5-4682-98e8-66319de27692 None None] MFA Rules not processed for user `15d2106935444c97b1e8c50131d3171f`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:23:59.585212 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-702641cd-38e5-4682-98e8-66319de27692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:23:59.592192 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1374/2748] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:23:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:24:00.150345 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210ae65e-806b-4ac9-94ef-78af549c3310 service cinder] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:24:00.151768 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-210ae65e-806b-4ac9-94ef-78af549c3310 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:24:00.158438 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:24:00.232797 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210ae65e-806b-4ac9-94ef-78af549c3310 service cinder] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:24:00.238370 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f9bcad-dba8-4b98-b798-4cb7de3e6008 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '680d62a67ae2456f8c129e26728c3297', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:24:00.238995 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f9bcad-dba8-4b98-b798-4cb7de3e6008 service placement] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:24:00.239275 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f9bcad-dba8-4b98-b798-4cb7de3e6008 service placement] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:24:00.239546 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f9bcad-dba8-4b98-b798-4cb7de3e6008 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:24:00.241316 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-81f9bcad-dba8-4b98-b798-4cb7de3e6008 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:24:00.252995 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f9bcad-dba8-4b98-b798-4cb7de3e6008 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:24:00.255264 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f9bcad-dba8-4b98-b798-4cb7de3e6008 service placement] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:24:00.265262 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1375/2749] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:24:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:24:01.735561 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702641cd-38e5-4682-98e8-66319de27692 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:24:01.736977 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-702641cd-38e5-4682-98e8-66319de27692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:24:01.747835 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:24:01.752259 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702641cd-38e5-4682-98e8-66319de27692 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:24:01.757883 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3fd3ed-c2df-4f91-94c7-4159da31e6fd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '680d62a67ae2456f8c129e26728c3297', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:24:01.761342 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3fd3ed-c2df-4f91-94c7-4159da31e6fd service placement] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:24:01.761342 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3fd3ed-c2df-4f91-94c7-4159da31e6fd service placement] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:24:01.761342 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3fd3ed-c2df-4f91-94c7-4159da31e6fd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:24:01.761342 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-2d3fd3ed-c2df-4f91-94c7-4159da31e6fd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:24:01.771893 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3fd3ed-c2df-4f91-94c7-4159da31e6fd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:24:01.773476 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3fd3ed-c2df-4f91-94c7-4159da31e6fd service placement] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:24:01.783455 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1375/2750] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:24:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:24:10.289048 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=30 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:24:11.871457 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=4 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:24:16.979369 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbbc714-bf0e-4e35-b119-fe5e08ca0184 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:24:16.979829 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbbc714-bf0e-4e35-b119-fe5e08ca0184 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:24:16.979951 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbbc714-bf0e-4e35-b119-fe5e08ca0184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:24:16.998766 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-4dbbc714-bf0e-4e35-b119-fe5e08ca0184 None None] MFA Rules not processed for user `d1b26ba592724dd389584bb800d877a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:24:16.999953 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-4dbbc714-bf0e-4e35-b119-fe5e08ca0184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:24:17.053416 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1376/2751] 10.222.0.18 () {58 vars in 1009 bytes} [Sat Aug 6 22:24:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:24:17.189411 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3fd3ed-c2df-4f91-94c7-4159da31e6fd service placement] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:24:17.191752 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-2d3fd3ed-c2df-4f91-94c7-4159da31e6fd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:24:17.270750 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3fd3ed-c2df-4f91-94c7-4159da31e6fd service placement] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:24:17.275567 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81d8f6e-1898-4498-82e4-5c7340e4cb45 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7fc9da7505744608dfd3bbd46a0a9d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:24:17.276158 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81d8f6e-1898-4498-82e4-5c7340e4cb45 service swift] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:24:17.276522 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81d8f6e-1898-4498-82e4-5c7340e4cb45 service swift] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:24:17.276796 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81d8f6e-1898-4498-82e4-5c7340e4cb45 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:24:17.284876 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81d8f6e-1898-4498-82e4-5c7340e4cb45 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:24:17.286433 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81d8f6e-1898-4498-82e4-5c7340e4cb45 service swift] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:24:17.294210 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1376/2752] 10.222.0.18 () {60 vars in 1384 bytes} [Sat Aug 6 22:24:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 22:24:18.131610 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61119b28-e753-46e0-a3c2-31f9ccbe547e None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:24:18.132081 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61119b28-e753-46e0-a3c2-31f9ccbe547e None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:24:18.132189 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61119b28-e753-46e0-a3c2-31f9ccbe547e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:24:18.151319 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-61119b28-e753-46e0-a3c2-31f9ccbe547e None None] MFA Rules not processed for user `d1b26ba592724dd389584bb800d877a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:24:18.152550 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-61119b28-e753-46e0-a3c2-31f9ccbe547e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:24:18.160196 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1377/2753] 10.222.0.18 () {58 vars in 967 bytes} [Sat Aug 6 22:24:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:24:18.172056 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81d8f6e-1898-4498-82e4-5c7340e4cb45 service swift] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:24:18.183680 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81d8f6e-1898-4498-82e4-5c7340e4cb45 service swift] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:24:18.189321 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba08ff13-3521-4a84-a9d5-6b7f5cd43063 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7fc9da7505744608dfd3bbd46a0a9d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:24:18.189971 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba08ff13-3521-4a84-a9d5-6b7f5cd43063 service swift] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:24:18.190549 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba08ff13-3521-4a84-a9d5-6b7f5cd43063 service swift] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:24:18.190662 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba08ff13-3521-4a84-a9d5-6b7f5cd43063 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:24:18.199555 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba08ff13-3521-4a84-a9d5-6b7f5cd43063 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:24:18.201217 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba08ff13-3521-4a84-a9d5-6b7f5cd43063 service swift] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:24:18.210091 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1377/2754] 10.222.0.18 () {60 vars in 1384 bytes} [Sat Aug 6 22:24:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 22:24:18.786921 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25f12ac-8e87-4621-a942-180dbe843e2d None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:24:18.787206 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25f12ac-8e87-4621-a942-180dbe843e2d None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:24:18.787510 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25f12ac-8e87-4621-a942-180dbe843e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:24:18.806787 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-a25f12ac-8e87-4621-a942-180dbe843e2d None None] MFA Rules not processed for user `d1b26ba592724dd389584bb800d877a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:24:18.809534 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-a25f12ac-8e87-4621-a942-180dbe843e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:24:18.815157 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1378/2755] 10.222.0.18 () {58 vars in 1009 bytes} [Sat Aug 6 22:24:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:24:18.827138 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba08ff13-3521-4a84-a9d5-6b7f5cd43063 service swift] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:24:18.838239 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba08ff13-3521-4a84-a9d5-6b7f5cd43063 service swift] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:24:18.843278 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8477b182-fec2-47e5-894f-70f443cc5ec3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7fc9da7505744608dfd3bbd46a0a9d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:24:18.844795 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8477b182-fec2-47e5-894f-70f443cc5ec3 service swift] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:24:18.844795 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8477b182-fec2-47e5-894f-70f443cc5ec3 service swift] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:24:18.844795 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8477b182-fec2-47e5-894f-70f443cc5ec3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:24:18.853472 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8477b182-fec2-47e5-894f-70f443cc5ec3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:24:18.855044 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8477b182-fec2-47e5-894f-70f443cc5ec3 service swift] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:24:18.863302 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1378/2756] 10.222.0.18 () {60 vars in 1384 bytes} [Sat Aug 6 22:24:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 22:24:19.481448 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb985e6-f813-4b05-9d48-4cb74c931408 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:24:19.481935 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb985e6-f813-4b05-9d48-4cb74c931408 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:24:19.482037 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb985e6-f813-4b05-9d48-4cb74c931408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:24:19.501881 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-5cb985e6-f813-4b05-9d48-4cb74c931408 None None] MFA Rules not processed for user `d1b26ba592724dd389584bb800d877a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:24:19.503131 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-5cb985e6-f813-4b05-9d48-4cb74c931408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:24:19.510180 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1379/2757] 10.222.0.18 () {58 vars in 967 bytes} [Sat Aug 6 22:24:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:24:19.522255 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8477b182-fec2-47e5-894f-70f443cc5ec3 service swift] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:24:19.532615 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8477b182-fec2-47e5-894f-70f443cc5ec3 service swift] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:24:19.538178 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149fdb00-e901-493e-8d28-6ab93b7557b3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7fc9da7505744608dfd3bbd46a0a9d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:24:19.538788 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149fdb00-e901-493e-8d28-6ab93b7557b3 service swift] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:24:19.539062 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149fdb00-e901-493e-8d28-6ab93b7557b3 service swift] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:24:19.539330 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149fdb00-e901-493e-8d28-6ab93b7557b3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:24:19.547293 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149fdb00-e901-493e-8d28-6ab93b7557b3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:24:19.548939 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149fdb00-e901-493e-8d28-6ab93b7557b3 service swift] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:24:19.557728 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1379/2758] 10.222.0.18 () {60 vars in 1384 bytes} [Sat Aug 6 22:24:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 22:24:22.242257 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb985e6-f813-4b05-9d48-4cb74c931408 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:24:22.303792 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb985e6-f813-4b05-9d48-4cb74c931408 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:24:22.308237 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b7bd91-8b20-42d5-b775-8b5f12d37111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:24:22.308879 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b7bd91-8b20-42d5-b775-8b5f12d37111 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:24:22.309154 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b7bd91-8b20-42d5-b775-8b5f12d37111 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:24:22.309421 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b7bd91-8b20-42d5-b775-8b5f12d37111 admin admin] PATH_INFO: `/v3/users/15d2106935444c97b1e8c50131d3171f` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:24:22.310532 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b7bd91-8b20-42d5-b775-8b5f12d37111 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15d2106935444c97b1e8c50131d3171f)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:24:22.312463 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b7bd91-8b20-42d5-b775-8b5f12d37111 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:24:22.492742 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-90b7bd91-8b20-42d5-b775-8b5f12d37111 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15d2106935444c97b1e8c50131d3171f'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:24:22.503626 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-90b7bd91-8b20-42d5-b775-8b5f12d37111 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15d2106935444c97b1e8c50131d3171f'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:24:22.510872 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-90b7bd91-8b20-42d5-b775-8b5f12d37111 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15d2106935444c97b1e8c50131d3171f'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:24:22.511664 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-90b7bd91-8b20-42d5-b775-8b5f12d37111 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15d2106935444c97b1e8c50131d3171f'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:24:22.512276 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-90b7bd91-8b20-42d5-b775-8b5f12d37111 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15d2106935444c97b1e8c50131d3171f'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:24:22.594021 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1380/2759] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:24:22 2022] DELETE /identity/v3/users/15d2106935444c97b1e8c50131d3171f => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:24:22.599712 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f560d0-c405-4bac-8dfa-f7d3c13827d7 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:24:22.600211 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f560d0-c405-4bac-8dfa-f7d3c13827d7 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:24:22.602823 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f560d0-c405-4bac-8dfa-f7d3c13827d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:24:22.619257 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-60f560d0-c405-4bac-8dfa-f7d3c13827d7 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:24:22.620481 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-60f560d0-c405-4bac-8dfa-f7d3c13827d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:24:22.677858 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1380/2760] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:24:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:24:22.687365 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b7bd91-8b20-42d5-b775-8b5f12d37111 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:24:22.688771 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-90b7bd91-8b20-42d5-b775-8b5f12d37111 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:24:22.695303 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:24:22.745449 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b7bd91-8b20-42d5-b775-8b5f12d37111 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:24:22.748936 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9f3c99-d4bf-4fb1-a0df-3876b43ac729 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:24:22.749520 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f3c99-d4bf-4fb1-a0df-3876b43ac729 service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:24:22.749799 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f3c99-d4bf-4fb1-a0df-3876b43ac729 service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:24:22.750064 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f3c99-d4bf-4fb1-a0df-3876b43ac729 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:24:22.759139 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9f3c99-d4bf-4fb1-a0df-3876b43ac729 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:24:22.760592 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9f3c99-d4bf-4fb1-a0df-3876b43ac729 service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:24:22.769690 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1381/2761] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:24:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:24:23.015207 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f560d0-c405-4bac-8dfa-f7d3c13827d7 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:24:23.016763 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-60f560d0-c405-4bac-8dfa-f7d3c13827d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:24:23.029870 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f560d0-c405-4bac-8dfa-f7d3c13827d7 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:24:23.034625 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26d6a53-436c-40e1-b8a8-61c03d02d760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:24:23.035261 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26d6a53-436c-40e1-b8a8-61c03d02d760 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:24:23.035450 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26d6a53-436c-40e1-b8a8-61c03d02d760 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:24:23.035782 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26d6a53-436c-40e1-b8a8-61c03d02d760 admin admin] PATH_INFO: `/v3/projects/676d7214383f4677989de5282569bd67` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:24:23.037023 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26d6a53-436c-40e1-b8a8-61c03d02d760 admin admin] RBAC: Authorizing `identity:delete_project(project_id=676d7214383f4677989de5282569bd67)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:24:23.038925 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26d6a53-436c-40e1-b8a8-61c03d02d760 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:24:23.154279 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:24:23.163794 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-b26d6a53-436c-40e1-b8a8-61c03d02d760 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '676d7214383f4677989de5282569bd67', 'request_id': 'req-b26d6a53-436c-40e1-b8a8-61c03d02d760'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:24:23.241937 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-b26d6a53-436c-40e1-b8a8-61c03d02d760 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '676d7214383f4677989de5282569bd67', 'request_id': 'req-b26d6a53-436c-40e1-b8a8-61c03d02d760'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:24:23.247738 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-b26d6a53-436c-40e1-b8a8-61c03d02d760 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '676d7214383f4677989de5282569bd67', 'request_id': 'req-b26d6a53-436c-40e1-b8a8-61c03d02d760'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:24:23.259586 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-b26d6a53-436c-40e1-b8a8-61c03d02d760 admin admin] The token cache is being invalidate because project 676d7214383f4677989de5282569bd67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:24:23.259870 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-b26d6a53-436c-40e1-b8a8-61c03d02d760 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:24:23.260522 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-b26d6a53-436c-40e1-b8a8-61c03d02d760 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:24:23.262075 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1381/2762] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:24:23 2022] DELETE /identity/v3/projects/676d7214383f4677989de5282569bd67 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:24:32.782236 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=24 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:24:33.256500 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:DELETE=5,keystone:SELECT=11,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:24:54.613883 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1353e66c-6a58-4f55-ac86-12611ee32e33 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:24:54.614475 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1353e66c-6a58-4f55-ac86-12611ee32e33 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:24:54.614475 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1353e66c-6a58-4f55-ac86-12611ee32e33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:24:54.633352 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-1353e66c-6a58-4f55-ac86-12611ee32e33 None None] MFA Rules not processed for user `2dae6a7ec4404626b75a6702386d091b`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:24:54.634537 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-1353e66c-6a58-4f55-ac86-12611ee32e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:24:54.687679 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1382/2763] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:24:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:24:54.996077 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26d6a53-436c-40e1-b8a8-61c03d02d760 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:24:54.997603 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-b26d6a53-436c-40e1-b8a8-61c03d02d760 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:24:55.055042 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26d6a53-436c-40e1-b8a8-61c03d02d760 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:24:55.058583 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e61a0e-aa8d-4e31-b3c0-27c9a08ecaad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:24:55.059176 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e61a0e-aa8d-4e31-b3c0-27c9a08ecaad service neutron] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:24:55.059450 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e61a0e-aa8d-4e31-b3c0-27c9a08ecaad service neutron] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:24:55.059718 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e61a0e-aa8d-4e31-b3c0-27c9a08ecaad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:24:55.061421 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-61e61a0e-aa8d-4e31-b3c0-27c9a08ecaad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:24:55.071390 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e61a0e-aa8d-4e31-b3c0-27c9a08ecaad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:24:55.072826 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e61a0e-aa8d-4e31-b3c0-27c9a08ecaad service neutron] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:24:55.080723 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1382/2764] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:24:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:25:04.681078 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=25 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:25:05.077267 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=26 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:25:20.018580 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1353e66c-6a58-4f55-ac86-12611ee32e33 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:25:20.020731 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-1353e66c-6a58-4f55-ac86-12611ee32e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:25:20.094550 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1353e66c-6a58-4f55-ac86-12611ee32e33 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:25:20.100349 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbfebaf-9cb3-4ec9-a9bb-4c4943d89fb1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a1319d33e9144859c6010843be312a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:25:20.101007 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbfebaf-9cb3-4ec9-a9bb-4c4943d89fb1 service nova] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:25:20.101285 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbfebaf-9cb3-4ec9-a9bb-4c4943d89fb1 service nova] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:25:20.101555 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbfebaf-9cb3-4ec9-a9bb-4c4943d89fb1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:25:20.103453 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-6cbfebaf-9cb3-4ec9-a9bb-4c4943d89fb1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:25:20.116206 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbfebaf-9cb3-4ec9-a9bb-4c4943d89fb1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:25:20.118028 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbfebaf-9cb3-4ec9-a9bb-4c4943d89fb1 service nova] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:25:20.130037 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1383/2765] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:25:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:25:30.124156 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=34 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:25:31.624847 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e61a0e-aa8d-4e31-b3c0-27c9a08ecaad service neutron] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:25:31.626274 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-61e61a0e-aa8d-4e31-b3c0-27c9a08ecaad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:25:31.683124 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e61a0e-aa8d-4e31-b3c0-27c9a08ecaad service neutron] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:25:31.686923 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53765fff-8d01-4257-abb2-498010da29f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa71c71de263478da234f3c5716a6e5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:25:31.688071 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53765fff-8d01-4257-abb2-498010da29f7 service cinder] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:25:31.688354 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53765fff-8d01-4257-abb2-498010da29f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:25:31.691374 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53765fff-8d01-4257-abb2-498010da29f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:25:31.691374 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-53765fff-8d01-4257-abb2-498010da29f7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:25:31.704264 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53765fff-8d01-4257-abb2-498010da29f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:25:31.705656 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53765fff-8d01-4257-abb2-498010da29f7 service cinder] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:25:31.718707 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1383/2766] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:25:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:25:35.705787 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbfebaf-9cb3-4ec9-a9bb-4c4943d89fb1 service nova] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:25:35.715596 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbfebaf-9cb3-4ec9-a9bb-4c4943d89fb1 service nova] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:25:35.719180 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822599af-be49-4e75-9b50-10c38048df20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:25:35.719765 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822599af-be49-4e75-9b50-10c38048df20 service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:25:35.720039 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822599af-be49-4e75-9b50-10c38048df20 service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:25:35.720307 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822599af-be49-4e75-9b50-10c38048df20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:25:35.722315 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-822599af-be49-4e75-9b50-10c38048df20 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:25:35.735274 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822599af-be49-4e75-9b50-10c38048df20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:25:35.736724 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822599af-be49-4e75-9b50-10c38048df20 service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:25:35.747378 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1384/2767] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:25:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:25:41.712372 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=26 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:25:45.742645 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=6 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:26:41.964373 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53765fff-8d01-4257-abb2-498010da29f7 service cinder] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:26:41.965876 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-53765fff-8d01-4257-abb2-498010da29f7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:26:41.977815 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53765fff-8d01-4257-abb2-498010da29f7 service cinder] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:26:41.981638 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfb4dca-bd62-49b3-9bd7-316a07d6d381 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:26:41.982255 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfb4dca-bd62-49b3-9bd7-316a07d6d381 service neutron] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:26:41.982528 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfb4dca-bd62-49b3-9bd7-316a07d6d381 service neutron] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:26:41.982797 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfb4dca-bd62-49b3-9bd7-316a07d6d381 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:26:41.984487 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-0dfb4dca-bd62-49b3-9bd7-316a07d6d381 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:26:41.993824 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfb4dca-bd62-49b3-9bd7-316a07d6d381 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:26:41.995376 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfb4dca-bd62-49b3-9bd7-316a07d6d381 service neutron] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:26:42.003444 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1384/2768] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:26:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:26:51.999730 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=6 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:26:53.117773 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1a1caf-419f-4e28-a2c9-ad2a4b406ffd None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:26:53.118309 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1a1caf-419f-4e28-a2c9-ad2a4b406ffd None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:26:53.118419 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1a1caf-419f-4e28-a2c9-ad2a4b406ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:26:53.138529 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-cb1a1caf-419f-4e28-a2c9-ad2a4b406ffd None None] MFA Rules not processed for user `563808cf212c40e2bd7f3ace5d1aec03`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:26:53.139715 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-cb1a1caf-419f-4e28-a2c9-ad2a4b406ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:26:53.194832 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1385/2769] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:26:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:27:01.572307 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfb4dca-bd62-49b3-9bd7-316a07d6d381 service neutron] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:27:01.574519 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-0dfb4dca-bd62-49b3-9bd7-316a07d6d381 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:27:01.603313 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfb4dca-bd62-49b3-9bd7-316a07d6d381 service neutron] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:27:01.609831 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a51c20-4fb0-4815-add2-ae7a80838c7c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a1319d33e9144859c6010843be312a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:27:01.610625 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a51c20-4fb0-4815-add2-ae7a80838c7c service nova] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:27:01.610970 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a51c20-4fb0-4815-add2-ae7a80838c7c service nova] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:27:01.611307 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a51c20-4fb0-4815-add2-ae7a80838c7c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:27:01.613480 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-04a51c20-4fb0-4815-add2-ae7a80838c7c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:27:01.635610 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a51c20-4fb0-4815-add2-ae7a80838c7c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:27:01.637336 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a51c20-4fb0-4815-add2-ae7a80838c7c service nova] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:27:01.650953 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1385/2770] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:27:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 22:27:03.191589 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=25 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:27:08.850789 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1a1caf-419f-4e28-a2c9-ad2a4b406ffd None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:27:08.852275 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-cb1a1caf-419f-4e28-a2c9-ad2a4b406ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:27:08.864614 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1a1caf-419f-4e28-a2c9-ad2a4b406ffd None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:27:08.868164 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a503233-7233-4321-9df6-baf7cf9ed70b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa71c71de263478da234f3c5716a6e5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:27:08.868736 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a503233-7233-4321-9df6-baf7cf9ed70b service cinder] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:27:08.869045 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a503233-7233-4321-9df6-baf7cf9ed70b service cinder] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:27:08.873246 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a503233-7233-4321-9df6-baf7cf9ed70b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:27:08.873246 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-5a503233-7233-4321-9df6-baf7cf9ed70b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:27:08.882398 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a503233-7233-4321-9df6-baf7cf9ed70b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:27:08.883963 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a503233-7233-4321-9df6-baf7cf9ed70b service cinder] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:27:08.892993 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1386/2771] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:27:08 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:27:11.644138 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=6 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:27:18.888863 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=6 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:27:20.477033 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee84112c-49dc-4d46-a6e2-870c449e53f1 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:27:20.477549 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee84112c-49dc-4d46-a6e2-870c449e53f1 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:27:20.477667 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee84112c-49dc-4d46-a6e2-870c449e53f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:27:20.497746 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-ee84112c-49dc-4d46-a6e2-870c449e53f1 None None] MFA Rules not processed for user `2dae6a7ec4404626b75a6702386d091b`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:27:20.498935 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-ee84112c-49dc-4d46-a6e2-870c449e53f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:27:20.505908 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1386/2772] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:27:20 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:27:20.822183 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a503233-7233-4321-9df6-baf7cf9ed70b service cinder] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:27:20.832062 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a503233-7233-4321-9df6-baf7cf9ed70b service cinder] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:27:20.835567 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482b7405-04eb-4de0-8cc6-426d7cc6b8ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:27:20.836148 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482b7405-04eb-4de0-8cc6-426d7cc6b8ef service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:27:20.836464 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482b7405-04eb-4de0-8cc6-426d7cc6b8ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:27:20.836739 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482b7405-04eb-4de0-8cc6-426d7cc6b8ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:27:20.838444 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-482b7405-04eb-4de0-8cc6-426d7cc6b8ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:27:20.848514 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482b7405-04eb-4de0-8cc6-426d7cc6b8ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:27:20.849912 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482b7405-04eb-4de0-8cc6-426d7cc6b8ef service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:27:20.857807 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1387/2773] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:27:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:27:30.493098 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=3 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:27:30.854120 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=6 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:27:35.165209 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee84112c-49dc-4d46-a6e2-870c449e53f1 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:27:35.178815 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee84112c-49dc-4d46-a6e2-870c449e53f1 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:27:35.184300 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2143c753-2075-4cee-afa5-34c566b51274 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a1319d33e9144859c6010843be312a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:27:35.184925 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2143c753-2075-4cee-afa5-34c566b51274 service nova] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:27:35.185203 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2143c753-2075-4cee-afa5-34c566b51274 service nova] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:27:35.185472 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2143c753-2075-4cee-afa5-34c566b51274 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:27:35.187194 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-2143c753-2075-4cee-afa5-34c566b51274 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:27:35.199404 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2143c753-2075-4cee-afa5-34c566b51274 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:27:35.203141 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2143c753-2075-4cee-afa5-34c566b51274 service nova] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:27:35.212813 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1387/2774] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:27:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:27:38.086427 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482b7405-04eb-4de0-8cc6-426d7cc6b8ef service neutron] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:27:38.095450 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482b7405-04eb-4de0-8cc6-426d7cc6b8ef service neutron] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:27:38.099007 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b8295b-cffa-4edc-8044-5aa77f21cdd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:27:38.099593 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b8295b-cffa-4edc-8044-5aa77f21cdd2 service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:27:38.099869 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b8295b-cffa-4edc-8044-5aa77f21cdd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:27:38.100138 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b8295b-cffa-4edc-8044-5aa77f21cdd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:27:38.101889 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-46b8295b-cffa-4edc-8044-5aa77f21cdd2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:27:38.164105 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b8295b-cffa-4edc-8044-5aa77f21cdd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:27:38.165575 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b8295b-cffa-4edc-8044-5aa77f21cdd2 service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:27:38.175921 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1388/2775] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:27:38 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:27:45.208294 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=6 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:27:48.171397 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=30 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:28:26.185595 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2143c753-2075-4cee-afa5-34c566b51274 service nova] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:28:26.197647 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2143c753-2075-4cee-afa5-34c566b51274 service nova] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:28:26.203200 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ccc456-1955-49ad-b40d-faecfe140b55 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a1319d33e9144859c6010843be312a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:28:26.203787 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ccc456-1955-49ad-b40d-faecfe140b55 service nova] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:28:26.204239 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ccc456-1955-49ad-b40d-faecfe140b55 service nova] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:28:26.204621 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ccc456-1955-49ad-b40d-faecfe140b55 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:28:26.206322 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-75ccc456-1955-49ad-b40d-faecfe140b55 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:28:26.216655 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ccc456-1955-49ad-b40d-faecfe140b55 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:28:26.218211 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ccc456-1955-49ad-b40d-faecfe140b55 service nova] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:28:26.227623 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1388/2776] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:28:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:28:36.223528 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=6 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:29:02.870161 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b8295b-cffa-4edc-8044-5aa77f21cdd2 service neutron] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:02.906354 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b8295b-cffa-4edc-8044-5aa77f21cdd2 service neutron] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:02.909935 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaf8acb-226a-4a43-b2dc-458c1ef44af8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:02.910540 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaf8acb-226a-4a43-b2dc-458c1ef44af8 service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:02.910804 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaf8acb-226a-4a43-b2dc-458c1ef44af8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:02.911076 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaf8acb-226a-4a43-b2dc-458c1ef44af8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:02.913766 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-cbaf8acb-226a-4a43-b2dc-458c1ef44af8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:29:03.025309 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaf8acb-226a-4a43-b2dc-458c1ef44af8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:03.026663 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaf8acb-226a-4a43-b2dc-458c1ef44af8 service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:03.045472 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1389/2777] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:29:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:29:06.764758 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ccc456-1955-49ad-b40d-faecfe140b55 service nova] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:06.769411 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-75ccc456-1955-49ad-b40d-faecfe140b55 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:29:06.895669 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ccc456-1955-49ad-b40d-faecfe140b55 service nova] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:06.907153 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d58115-ab88-48ba-8925-33b52dd04eaf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '680d62a67ae2456f8c129e26728c3297', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:06.907611 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaf8acb-226a-4a43-b2dc-458c1ef44af8 service neutron] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:06.908117 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d58115-ab88-48ba-8925-33b52dd04eaf service placement] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:06.908117 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d58115-ab88-48ba-8925-33b52dd04eaf service placement] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:06.908334 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d58115-ab88-48ba-8925-33b52dd04eaf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:06.910226 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-d4d58115-ab88-48ba-8925-33b52dd04eaf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:29:06.910428 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-cbaf8acb-226a-4a43-b2dc-458c1ef44af8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:29:06.922832 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d58115-ab88-48ba-8925-33b52dd04eaf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:06.924127 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaf8acb-226a-4a43-b2dc-458c1ef44af8 service neutron] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:06.924516 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d58115-ab88-48ba-8925-33b52dd04eaf service placement] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:06.929768 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fb4dca-a56a-47e0-8462-8c57da680644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:06.930478 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fb4dca-a56a-47e0-8462-8c57da680644 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:06.930760 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fb4dca-a56a-47e0-8462-8c57da680644 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:06.931251 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fb4dca-a56a-47e0-8462-8c57da680644 admin admin] PATH_INFO: `/v3/users/563808cf212c40e2bd7f3ace5d1aec03` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:06.932572 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fb4dca-a56a-47e0-8462-8c57da680644 admin admin] RBAC: Authorizing `identity:delete_user(user_id=563808cf212c40e2bd7f3ace5d1aec03)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:06.934831 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fb4dca-a56a-47e0-8462-8c57da680644 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:06.935348 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1389/2778] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:29:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:29:07.092276 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-d5fb4dca-a56a-47e0-8462-8c57da680644 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '563808cf212c40e2bd7f3ace5d1aec03'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:07.104829 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-d5fb4dca-a56a-47e0-8462-8c57da680644 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '563808cf212c40e2bd7f3ace5d1aec03'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:07.105981 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:29:07.112002 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-d5fb4dca-a56a-47e0-8462-8c57da680644 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '563808cf212c40e2bd7f3ace5d1aec03'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:07.112704 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-d5fb4dca-a56a-47e0-8462-8c57da680644 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '563808cf212c40e2bd7f3ace5d1aec03'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:07.113315 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-d5fb4dca-a56a-47e0-8462-8c57da680644 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '563808cf212c40e2bd7f3ace5d1aec03'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:07.186485 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1390/2779] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:29:06 2022] DELETE /identity/v3/users/563808cf212c40e2bd7f3ace5d1aec03 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:29:07.429427 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d58115-ab88-48ba-8925-33b52dd04eaf service placement] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:07.440146 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d58115-ab88-48ba-8925-33b52dd04eaf service placement] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:07.497762 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbc4a8d-db03-49b3-8665-98b7d7fdebef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:07.498320 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc4a8d-db03-49b3-8665-98b7d7fdebef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:07.498597 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc4a8d-db03-49b3-8665-98b7d7fdebef admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:07.498868 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc4a8d-db03-49b3-8665-98b7d7fdebef admin admin] PATH_INFO: `/v3/projects/8abd60d3224e4e2995d55417a73ffe58` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:07.499977 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbc4a8d-db03-49b3-8665-98b7d7fdebef admin admin] RBAC: Authorizing `identity:delete_project(project_id=8abd60d3224e4e2995d55417a73ffe58)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:07.501898 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbc4a8d-db03-49b3-8665-98b7d7fdebef admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:07.592184 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:29:07.604721 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-ddbc4a8d-db03-49b3-8665-98b7d7fdebef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8abd60d3224e4e2995d55417a73ffe58', 'request_id': 'req-ddbc4a8d-db03-49b3-8665-98b7d7fdebef'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:07.658880 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-ddbc4a8d-db03-49b3-8665-98b7d7fdebef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8abd60d3224e4e2995d55417a73ffe58', 'request_id': 'req-ddbc4a8d-db03-49b3-8665-98b7d7fdebef'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:07.665012 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-ddbc4a8d-db03-49b3-8665-98b7d7fdebef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8abd60d3224e4e2995d55417a73ffe58', 'request_id': 'req-ddbc4a8d-db03-49b3-8665-98b7d7fdebef'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:07.676000 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-ddbc4a8d-db03-49b3-8665-98b7d7fdebef admin admin] The token cache is being invalidate because project 8abd60d3224e4e2995d55417a73ffe58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:29:07.676285 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-ddbc4a8d-db03-49b3-8665-98b7d7fdebef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:07.676941 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-ddbc4a8d-db03-49b3-8665-98b7d7fdebef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:07.679048 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1390/2780] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:29:07 2022] DELETE /identity/v3/projects/8abd60d3224e4e2995d55417a73ffe58 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:29:07.683907 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69222269-13c8-44f8-be3f-10a38dda7748 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:07.684193 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69222269-13c8-44f8-be3f-10a38dda7748 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:07.684527 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69222269-13c8-44f8-be3f-10a38dda7748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:07.714729 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-69222269-13c8-44f8-be3f-10a38dda7748 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:29:07.715954 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-69222269-13c8-44f8-be3f-10a38dda7748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:29:07.784303 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1391/2781] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:29:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:29:07.793374 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a090e3f-6487-4731-bfd1-95bb7ac400ea None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:07.793900 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a090e3f-6487-4731-bfd1-95bb7ac400ea None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:07.794104 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a090e3f-6487-4731-bfd1-95bb7ac400ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:07.814737 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-2a090e3f-6487-4731-bfd1-95bb7ac400ea None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:29:07.816136 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-2a090e3f-6487-4731-bfd1-95bb7ac400ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:29:07.824492 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1391/2782] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:29:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:29:07.828776 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69222269-13c8-44f8-be3f-10a38dda7748 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:07.838855 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69222269-13c8-44f8-be3f-10a38dda7748 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:07.843563 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20adc452-467a-4e43-9d9c-ca2e9dfbe9a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:07.844159 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20adc452-467a-4e43-9d9c-ca2e9dfbe9a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:07.844465 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20adc452-467a-4e43-9d9c-ca2e9dfbe9a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:07.844987 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20adc452-467a-4e43-9d9c-ca2e9dfbe9a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:07.845629 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20adc452-467a-4e43-9d9c-ca2e9dfbe9a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 22:29:07.845923 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20adc452-467a-4e43-9d9c-ca2e9dfbe9a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:07.847518 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20adc452-467a-4e43-9d9c-ca2e9dfbe9a0 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:07.859937 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1392/2783] 10.222.0.18 () {58 vars in 1153 bytes} [Sat Aug 6 22:29:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 22:29:07.864276 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a090e3f-6487-4731-bfd1-95bb7ac400ea None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:07.874690 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a090e3f-6487-4731-bfd1-95bb7ac400ea None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:07.879464 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcf036b-72c0-4765-b6d5-47635dd4b979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:07.880030 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcf036b-72c0-4765-b6d5-47635dd4b979 admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:07.880309 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcf036b-72c0-4765-b6d5-47635dd4b979 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:07.880617 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcf036b-72c0-4765-b6d5-47635dd4b979 admin admin] PATH_INFO: `/v3/projects` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:07.881349 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcf036b-72c0-4765-b6d5-47635dd4b979 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:07.883269 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcf036b-72c0-4765-b6d5-47635dd4b979 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:07.986844 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1392/2784] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:29:07 2022] POST /identity/v3/projects => generated 405 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 22:29:07.991150 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20adc452-467a-4e43-9d9c-ca2e9dfbe9a0 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:08.001183 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20adc452-467a-4e43-9d9c-ca2e9dfbe9a0 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:08.055813 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20d323c-0a92-4705-8f96-935c90fe9b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:08.056321 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20d323c-0a92-4705-8f96-935c90fe9b90 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:08.056624 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20d323c-0a92-4705-8f96-935c90fe9b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:08.056895 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20d323c-0a92-4705-8f96-935c90fe9b90 admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:08.057514 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20d323c-0a92-4705-8f96-935c90fe9b90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:08.059404 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20d323c-0a92-4705-8f96-935c90fe9b90 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:08.186564 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1393/2785] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:29:07 2022] POST /identity/v3/users => generated 303 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:29:08.190731 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcf036b-72c0-4765-b6d5-47635dd4b979 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:08.201130 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcf036b-72c0-4765-b6d5-47635dd4b979 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:08.205667 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05bb964-001f-482c-bab6-cc3df385a36d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:08.206253 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05bb964-001f-482c-bab6-cc3df385a36d admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:08.206525 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05bb964-001f-482c-bab6-cc3df385a36d admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:08.206790 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05bb964-001f-482c-bab6-cc3df385a36d admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:08.207420 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05bb964-001f-482c-bab6-cc3df385a36d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:08.209546 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05bb964-001f-482c-bab6-cc3df385a36d admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:08.219243 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1393/2786] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:29:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:29:08.223508 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20d323c-0a92-4705-8f96-935c90fe9b90 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:08.234535 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20d323c-0a92-4705-8f96-935c90fe9b90 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:08.239199 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2073ef59-df5e-4d4a-ade6-28391691b678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:08.239789 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2073ef59-df5e-4d4a-ade6-28391691b678 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:08.240068 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2073ef59-df5e-4d4a-ade6-28391691b678 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:08.240459 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2073ef59-df5e-4d4a-ade6-28391691b678 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:08.241285 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2073ef59-df5e-4d4a-ade6-28391691b678 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:08.242872 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2073ef59-df5e-4d4a-ade6-28391691b678 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:08.253157 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1394/2787] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:29:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:29:08.257859 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05bb964-001f-482c-bab6-cc3df385a36d admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:08.267622 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05bb964-001f-482c-bab6-cc3df385a36d admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:08.272908 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3173133a-ff1f-4803-b795-a11073a8e7c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:08.273549 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3173133a-ff1f-4803-b795-a11073a8e7c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:08.273831 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3173133a-ff1f-4803-b795-a11073a8e7c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:08.274103 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3173133a-ff1f-4803-b795-a11073a8e7c2 admin admin] PATH_INFO: `/v3/projects/938d933d975c495cae245fa39f66dd53/users/6818c41d8baa43baa2c42682bb9b88c4/roles/56e4a96094e9434cbbafb22579c034f5` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:08.288630 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3173133a-ff1f-4803-b795-a11073a8e7c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=938d933d975c495cae245fa39f66dd53, user_id=6818c41d8baa43baa2c42682bb9b88c4, role_id=56e4a96094e9434cbbafb22579c034f5)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:08.291281 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3173133a-ff1f-4803-b795-a11073a8e7c2 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:08.356509 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1394/2788] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:29:08 2022] PUT /identity/v3/projects/938d933d975c495cae245fa39f66dd53/users/6818c41d8baa43baa2c42682bb9b88c4/roles/56e4a96094e9434cbbafb22579c034f5 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:29:08.361684 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c939c8c-e2c1-4470-952d-c26c6dab37c7 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:08.361969 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c939c8c-e2c1-4470-952d-c26c6dab37c7 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:08.362244 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c939c8c-e2c1-4470-952d-c26c6dab37c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:08.381548 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-4c939c8c-e2c1-4470-952d-c26c6dab37c7 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:29:08.383768 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-4c939c8c-e2c1-4470-952d-c26c6dab37c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:29:08.416614 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:29:08.441154 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1395/2789] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:29:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:29:08.451664 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3173133a-ff1f-4803-b795-a11073a8e7c2 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:08.453392 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-3173133a-ff1f-4803-b795-a11073a8e7c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:29:08.510785 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3173133a-ff1f-4803-b795-a11073a8e7c2 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:08.514464 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3094514f-19e7-4226-89d2-13e947f4bb25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:08.515057 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3094514f-19e7-4226-89d2-13e947f4bb25 service neutron] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:08.515331 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3094514f-19e7-4226-89d2-13e947f4bb25 service neutron] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:08.515595 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3094514f-19e7-4226-89d2-13e947f4bb25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:08.525098 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3094514f-19e7-4226-89d2-13e947f4bb25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:08.526508 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3094514f-19e7-4226-89d2-13e947f4bb25 service neutron] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:08.535659 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1395/2790] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:29:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:29:09.728066 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa476fa8-6aa1-41fb-9607-f3b5fb52d96d None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:09.734382 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa476fa8-6aa1-41fb-9607-f3b5fb52d96d None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:09.734382 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa476fa8-6aa1-41fb-9607-f3b5fb52d96d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:09.757165 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-fa476fa8-6aa1-41fb-9607-f3b5fb52d96d None None] MFA Rules not processed for user `6818c41d8baa43baa2c42682bb9b88c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:29:09.763058 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-fa476fa8-6aa1-41fb-9607-f3b5fb52d96d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:29:09.826795 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1396/2791] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:29:09 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:29:09.831231 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3094514f-19e7-4226-89d2-13e947f4bb25 service neutron] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:09.841945 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3094514f-19e7-4226-89d2-13e947f4bb25 service neutron] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:09.850333 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75bfe84-0302-4ad7-a9d0-7fe0916754fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:09.850333 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75bfe84-0302-4ad7-a9d0-7fe0916754fe admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:09.850333 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75bfe84-0302-4ad7-a9d0-7fe0916754fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:09.850333 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75bfe84-0302-4ad7-a9d0-7fe0916754fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:09.850789 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75bfe84-0302-4ad7-a9d0-7fe0916754fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:09.850789 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75bfe84-0302-4ad7-a9d0-7fe0916754fe admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:09.930877 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1396/2792] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:29:09 2022] POST /identity/v3/projects => generated 403 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:29:09.934852 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa476fa8-6aa1-41fb-9607-f3b5fb52d96d None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:09.946356 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa476fa8-6aa1-41fb-9607-f3b5fb52d96d None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:10.000984 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca1c462-432e-46fe-bee5-a9c28c12bba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:10.001580 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca1c462-432e-46fe-bee5-a9c28c12bba4 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:10.001858 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca1c462-432e-46fe-bee5-a9c28c12bba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:10.002135 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca1c462-432e-46fe-bee5-a9c28c12bba4 admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:10.002758 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca1c462-432e-46fe-bee5-a9c28c12bba4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:10.004693 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca1c462-432e-46fe-bee5-a9c28c12bba4 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:10.107009 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1397/2793] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:29:09 2022] POST /identity/v3/users => generated 308 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:29:10.110996 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75bfe84-0302-4ad7-a9d0-7fe0916754fe admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:10.121505 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75bfe84-0302-4ad7-a9d0-7fe0916754fe admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:10.126074 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e49d59-cb3d-4607-8e40-b39c8778bb0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:10.126664 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e49d59-cb3d-4607-8e40-b39c8778bb0b admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:10.126940 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e49d59-cb3d-4607-8e40-b39c8778bb0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:10.127234 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e49d59-cb3d-4607-8e40-b39c8778bb0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:10.127893 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e49d59-cb3d-4607-8e40-b39c8778bb0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:10.129506 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e49d59-cb3d-4607-8e40-b39c8778bb0b admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:10.140756 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1397/2794] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:29:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:29:10.144951 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca1c462-432e-46fe-bee5-a9c28c12bba4 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:10.155675 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca1c462-432e-46fe-bee5-a9c28c12bba4 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:10.160163 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af1790f-b594-4079-adb4-665493647ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:10.160838 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af1790f-b594-4079-adb4-665493647ac3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:10.161116 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af1790f-b594-4079-adb4-665493647ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:10.161387 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af1790f-b594-4079-adb4-665493647ac3 admin admin] PATH_INFO: `/v3/projects/29171bfbacfd49148fed62f54146c9ca/users/1776367e0f9840598ad590b90aba359c/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:10.176582 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af1790f-b594-4079-adb4-665493647ac3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29171bfbacfd49148fed62f54146c9ca, user_id=1776367e0f9840598ad590b90aba359c, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:10.179697 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af1790f-b594-4079-adb4-665493647ac3 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:10.254666 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1398/2795] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:29:10 2022] PUT /identity/v3/projects/29171bfbacfd49148fed62f54146c9ca/users/1776367e0f9840598ad590b90aba359c/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:29:10.259276 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e49d59-cb3d-4607-8e40-b39c8778bb0b admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:10.269004 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e49d59-cb3d-4607-8e40-b39c8778bb0b admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:10.292766 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:29:10.325680 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa465689-d3fb-4a3b-b3cb-71f60fb51bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:10.326307 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa465689-d3fb-4a3b-b3cb-71f60fb51bad admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:10.326597 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa465689-d3fb-4a3b-b3cb-71f60fb51bad admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:10.327068 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa465689-d3fb-4a3b-b3cb-71f60fb51bad admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:10.327866 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa465689-d3fb-4a3b-b3cb-71f60fb51bad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:10.329925 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa465689-d3fb-4a3b-b3cb-71f60fb51bad admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:10.339457 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1398/2796] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:29:10 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:29:10.343642 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af1790f-b594-4079-adb4-665493647ac3 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:10.354382 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af1790f-b594-4079-adb4-665493647ac3 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:10.359200 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59fa6bb-401a-40d3-8ae3-a0e25b16f525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:10.359844 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59fa6bb-401a-40d3-8ae3-a0e25b16f525 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:10.360119 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59fa6bb-401a-40d3-8ae3-a0e25b16f525 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:10.360387 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59fa6bb-401a-40d3-8ae3-a0e25b16f525 admin admin] PATH_INFO: `/v3/projects/29171bfbacfd49148fed62f54146c9ca/users/1776367e0f9840598ad590b90aba359c/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:10.362584 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59fa6bb-401a-40d3-8ae3-a0e25b16f525 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29171bfbacfd49148fed62f54146c9ca, user_id=1776367e0f9840598ad590b90aba359c, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:10.365906 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59fa6bb-401a-40d3-8ae3-a0e25b16f525 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:10.376300 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1399/2797] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:29:10 2022] PUT /identity/v3/projects/29171bfbacfd49148fed62f54146c9ca/users/1776367e0f9840598ad590b90aba359c/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:29:11.570565 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d50b3fa-4eb3-41a7-86a6-a8a3df0be632 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:11.571105 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d50b3fa-4eb3-41a7-86a6-a8a3df0be632 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:11.571219 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d50b3fa-4eb3-41a7-86a6-a8a3df0be632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:11.603277 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-1d50b3fa-4eb3-41a7-86a6-a8a3df0be632 None None] MFA Rules not processed for user `1776367e0f9840598ad590b90aba359c`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:29:11.604430 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-1d50b3fa-4eb3-41a7-86a6-a8a3df0be632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:29:11.675513 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1399/2798] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:29:11 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:29:12.856432 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59fa6bb-401a-40d3-8ae3-a0e25b16f525 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:12.858132 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-a59fa6bb-401a-40d3-8ae3-a0e25b16f525 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:29:12.884885 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:29:12.928823 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59fa6bb-401a-40d3-8ae3-a0e25b16f525 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:12.934152 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbd2c92-9e73-44af-855e-319dd4242cd1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '680d62a67ae2456f8c129e26728c3297', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:12.934749 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbd2c92-9e73-44af-855e-319dd4242cd1 service placement] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:12.935032 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbd2c92-9e73-44af-855e-319dd4242cd1 service placement] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:12.935303 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbd2c92-9e73-44af-855e-319dd4242cd1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:12.937030 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-0fbd2c92-9e73-44af-855e-319dd4242cd1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:29:12.948461 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbd2c92-9e73-44af-855e-319dd4242cd1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:12.950088 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbd2c92-9e73-44af-855e-319dd4242cd1 service placement] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:12.959754 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1400/2799] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:29:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 22:29:12.999907 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d50b3fa-4eb3-41a7-86a6-a8a3df0be632 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:13.062163 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d50b3fa-4eb3-41a7-86a6-a8a3df0be632 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:13.066747 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec875dcd-8ba0-4091-847f-1b521f78f41c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:13.067368 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec875dcd-8ba0-4091-847f-1b521f78f41c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:13.067646 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec875dcd-8ba0-4091-847f-1b521f78f41c admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:13.067917 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec875dcd-8ba0-4091-847f-1b521f78f41c admin admin] PATH_INFO: `/v3/users/6818c41d8baa43baa2c42682bb9b88c4` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:13.069154 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec875dcd-8ba0-4091-847f-1b521f78f41c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6818c41d8baa43baa2c42682bb9b88c4)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:13.071049 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec875dcd-8ba0-4091-847f-1b521f78f41c admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:13.279998 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-ec875dcd-8ba0-4091-847f-1b521f78f41c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6818c41d8baa43baa2c42682bb9b88c4'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:13.281794 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:29:13.293540 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-ec875dcd-8ba0-4091-847f-1b521f78f41c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6818c41d8baa43baa2c42682bb9b88c4'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:13.363684 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-ec875dcd-8ba0-4091-847f-1b521f78f41c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6818c41d8baa43baa2c42682bb9b88c4'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:13.371127 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-ec875dcd-8ba0-4091-847f-1b521f78f41c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6818c41d8baa43baa2c42682bb9b88c4'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:13.374027 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-ec875dcd-8ba0-4091-847f-1b521f78f41c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6818c41d8baa43baa2c42682bb9b88c4'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:13.378233 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1400/2800] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:29:12 2022] DELETE /identity/v3/users/6818c41d8baa43baa2c42682bb9b88c4 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:29:13.616684 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbd2c92-9e73-44af-855e-319dd4242cd1 service placement] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:13.631965 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbd2c92-9e73-44af-855e-319dd4242cd1 service placement] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:13.684568 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692e465d-89a2-4747-b90c-5abc6e20d784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:13.685194 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e465d-89a2-4747-b90c-5abc6e20d784 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:13.685474 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e465d-89a2-4747-b90c-5abc6e20d784 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:13.685744 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e465d-89a2-4747-b90c-5abc6e20d784 admin admin] PATH_INFO: `/v3/projects/938d933d975c495cae245fa39f66dd53` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:13.686884 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692e465d-89a2-4747-b90c-5abc6e20d784 admin admin] RBAC: Authorizing `identity:delete_project(project_id=938d933d975c495cae245fa39f66dd53)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:13.688866 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692e465d-89a2-4747-b90c-5abc6e20d784 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:13.830664 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-692e465d-89a2-4747-b90c-5abc6e20d784 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '938d933d975c495cae245fa39f66dd53', 'request_id': 'req-692e465d-89a2-4747-b90c-5abc6e20d784'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:13.889377 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-692e465d-89a2-4747-b90c-5abc6e20d784 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '938d933d975c495cae245fa39f66dd53', 'request_id': 'req-692e465d-89a2-4747-b90c-5abc6e20d784'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:13.894940 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-692e465d-89a2-4747-b90c-5abc6e20d784 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '938d933d975c495cae245fa39f66dd53', 'request_id': 'req-692e465d-89a2-4747-b90c-5abc6e20d784'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:13.906516 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-692e465d-89a2-4747-b90c-5abc6e20d784 admin admin] The token cache is being invalidate because project 938d933d975c495cae245fa39f66dd53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:29:13.906801 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-692e465d-89a2-4747-b90c-5abc6e20d784 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:13.907426 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-692e465d-89a2-4747-b90c-5abc6e20d784 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:13.908966 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1401/2801] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:29:13 2022] DELETE /identity/v3/projects/938d933d975c495cae245fa39f66dd53 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:29:13.912959 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec875dcd-8ba0-4091-847f-1b521f78f41c admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:13.914389 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-ec875dcd-8ba0-4091-847f-1b521f78f41c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:29:13.928147 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec875dcd-8ba0-4091-847f-1b521f78f41c admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:13.984349 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191a6def-914b-439f-b750-7eaede2ca7a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:13.985008 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191a6def-914b-439f-b750-7eaede2ca7a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:13.985283 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191a6def-914b-439f-b750-7eaede2ca7a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:13.985553 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191a6def-914b-439f-b750-7eaede2ca7a2 admin admin] PATH_INFO: `/v3/users/1776367e0f9840598ad590b90aba359c` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:13.986674 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191a6def-914b-439f-b750-7eaede2ca7a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1776367e0f9840598ad590b90aba359c)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:13.988601 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191a6def-914b-439f-b750-7eaede2ca7a2 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:14.209422 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-191a6def-914b-439f-b750-7eaede2ca7a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1776367e0f9840598ad590b90aba359c'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:14.221329 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-191a6def-914b-439f-b750-7eaede2ca7a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1776367e0f9840598ad590b90aba359c'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:14.304715 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-191a6def-914b-439f-b750-7eaede2ca7a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1776367e0f9840598ad590b90aba359c'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:14.312092 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-191a6def-914b-439f-b750-7eaede2ca7a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1776367e0f9840598ad590b90aba359c'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:14.312786 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-191a6def-914b-439f-b750-7eaede2ca7a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1776367e0f9840598ad590b90aba359c'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:14.319096 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1401/2802] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:29:13 2022] DELETE /identity/v3/users/1776367e0f9840598ad590b90aba359c => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:29:14.623121 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692e465d-89a2-4747-b90c-5abc6e20d784 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:14.625756 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-692e465d-89a2-4747-b90c-5abc6e20d784 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:29:14.637665 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692e465d-89a2-4747-b90c-5abc6e20d784 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:14.679681 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:29:14.695464 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4200d7f-0416-45f5-9a63-e8ad92aab215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:14.696086 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4200d7f-0416-45f5-9a63-e8ad92aab215 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:14.699062 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4200d7f-0416-45f5-9a63-e8ad92aab215 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:14.699062 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4200d7f-0416-45f5-9a63-e8ad92aab215 admin admin] PATH_INFO: `/v3/projects/29171bfbacfd49148fed62f54146c9ca` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:14.699062 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4200d7f-0416-45f5-9a63-e8ad92aab215 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29171bfbacfd49148fed62f54146c9ca)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:14.699606 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4200d7f-0416-45f5-9a63-e8ad92aab215 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:14.797656 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-c4200d7f-0416-45f5-9a63-e8ad92aab215 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29171bfbacfd49148fed62f54146c9ca', 'request_id': 'req-c4200d7f-0416-45f5-9a63-e8ad92aab215'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:14.872313 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-c4200d7f-0416-45f5-9a63-e8ad92aab215 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29171bfbacfd49148fed62f54146c9ca', 'request_id': 'req-c4200d7f-0416-45f5-9a63-e8ad92aab215'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:14.878017 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-c4200d7f-0416-45f5-9a63-e8ad92aab215 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29171bfbacfd49148fed62f54146c9ca', 'request_id': 'req-c4200d7f-0416-45f5-9a63-e8ad92aab215'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:14.889336 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-c4200d7f-0416-45f5-9a63-e8ad92aab215 admin admin] The token cache is being invalidate because project 29171bfbacfd49148fed62f54146c9ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:29:14.889624 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-c4200d7f-0416-45f5-9a63-e8ad92aab215 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:14.890246 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-c4200d7f-0416-45f5-9a63-e8ad92aab215 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:14.891743 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1402/2803] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:29:14 2022] DELETE /identity/v3/projects/29171bfbacfd49148fed62f54146c9ca => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:29:14.898244 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d6037-6e71-46fe-abc5-78ed97713571 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:14.898528 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d6037-6e71-46fe-abc5-78ed97713571 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:14.898802 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d6037-6e71-46fe-abc5-78ed97713571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:14.918597 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-9e3d6037-6e71-46fe-abc5-78ed97713571 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:29:14.919787 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-9e3d6037-6e71-46fe-abc5-78ed97713571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:29:14.960820 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:29:14.978465 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1402/2804] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:29:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:29:14.986226 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db6c42-7130-4803-a33e-750a2bb13af4 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:14.986641 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db6c42-7130-4803-a33e-750a2bb13af4 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:14.986806 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db6c42-7130-4803-a33e-750a2bb13af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:15.007858 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-06db6c42-7130-4803-a33e-750a2bb13af4 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:29:15.009144 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-06db6c42-7130-4803-a33e-750a2bb13af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:29:15.017040 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1403/2805] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:29:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:29:15.021228 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3d6037-6e71-46fe-abc5-78ed97713571 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:15.031551 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3d6037-6e71-46fe-abc5-78ed97713571 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:15.036719 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a150fd6-87ec-4566-854b-33ea8cf32a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:15.037349 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a150fd6-87ec-4566-854b-33ea8cf32a02 admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:15.037590 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a150fd6-87ec-4566-854b-33ea8cf32a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:15.037860 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a150fd6-87ec-4566-854b-33ea8cf32a02 admin admin] PATH_INFO: `/v3/domains` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:15.038515 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a150fd6-87ec-4566-854b-33ea8cf32a02 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 22:29:15.038814 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a150fd6-87ec-4566-854b-33ea8cf32a02 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:15.040445 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a150fd6-87ec-4566-854b-33ea8cf32a02 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:15.051838 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1403/2806] 10.222.0.18 () {58 vars in 1153 bytes} [Sat Aug 6 22:29:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 22:29:15.056313 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06db6c42-7130-4803-a33e-750a2bb13af4 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:15.065440 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06db6c42-7130-4803-a33e-750a2bb13af4 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:15.070999 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8ec8bb-0915-4cd0-b66c-29b2bb09b8a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:15.073081 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8ec8bb-0915-4cd0-b66c-29b2bb09b8a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:15.073081 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8ec8bb-0915-4cd0-b66c-29b2bb09b8a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:15.073081 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8ec8bb-0915-4cd0-b66c-29b2bb09b8a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:15.073081 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8ec8bb-0915-4cd0-b66c-29b2bb09b8a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:15.074758 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8ec8bb-0915-4cd0-b66c-29b2bb09b8a4 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:15.159533 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1404/2807] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:29:15 2022] POST /identity/v3/projects => generated 369 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:29:15.163882 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a150fd6-87ec-4566-854b-33ea8cf32a02 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:15.173959 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a150fd6-87ec-4566-854b-33ea8cf32a02 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:15.230519 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f99476b-741a-4099-a438-3eb833891951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:15.231196 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f99476b-741a-4099-a438-3eb833891951 admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:15.231405 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f99476b-741a-4099-a438-3eb833891951 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:15.231671 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f99476b-741a-4099-a438-3eb833891951 admin admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:15.232292 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f99476b-741a-4099-a438-3eb833891951 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:15.234599 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f99476b-741a-4099-a438-3eb833891951 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:15.322030 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1404/2808] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:29:15 2022] POST /identity/v3/users => generated 285 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:29:15.326068 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8ec8bb-0915-4cd0-b66c-29b2bb09b8a4 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:15.336710 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8ec8bb-0915-4cd0-b66c-29b2bb09b8a4 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:15.341317 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8886a861-6802-4b3e-9177-5d5d7eddeb2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:15.341903 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8886a861-6802-4b3e-9177-5d5d7eddeb2a admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:15.342207 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8886a861-6802-4b3e-9177-5d5d7eddeb2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:15.342475 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8886a861-6802-4b3e-9177-5d5d7eddeb2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:15.343180 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8886a861-6802-4b3e-9177-5d5d7eddeb2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:15.345073 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8886a861-6802-4b3e-9177-5d5d7eddeb2a admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:15.356020 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1405/2809] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:29:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:29:15.360039 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f99476b-741a-4099-a438-3eb833891951 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:15.371188 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f99476b-741a-4099-a438-3eb833891951 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:15.375972 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6737a44d-4fea-4462-9608-2f3e410e3915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:15.376599 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6737a44d-4fea-4462-9608-2f3e410e3915 admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:15.376879 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6737a44d-4fea-4462-9608-2f3e410e3915 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:15.377239 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6737a44d-4fea-4462-9608-2f3e410e3915 admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:15.377876 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6737a44d-4fea-4462-9608-2f3e410e3915 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:15.379472 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6737a44d-4fea-4462-9608-2f3e410e3915 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:15.390124 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1405/2810] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:29:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:29:15.394324 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8886a861-6802-4b3e-9177-5d5d7eddeb2a admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:15.404862 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8886a861-6802-4b3e-9177-5d5d7eddeb2a admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:15.409510 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a8d46d-8798-4d90-bad1-720fd8e05a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:15.410205 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a8d46d-8798-4d90-bad1-720fd8e05a9e admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:15.410435 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a8d46d-8798-4d90-bad1-720fd8e05a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:15.410706 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a8d46d-8798-4d90-bad1-720fd8e05a9e admin admin] PATH_INFO: `/v3/projects/7f7b14c04c694733bb036c5c369b023a/users/ece311c6ff38448b9be3ede7c6dafb0a/roles/56e4a96094e9434cbbafb22579c034f5` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:15.425512 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a8d46d-8798-4d90-bad1-720fd8e05a9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f7b14c04c694733bb036c5c369b023a, user_id=ece311c6ff38448b9be3ede7c6dafb0a, role_id=56e4a96094e9434cbbafb22579c034f5)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:15.428325 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a8d46d-8798-4d90-bad1-720fd8e05a9e admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:15.544789 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1406/2811] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:29:15 2022] PUT /identity/v3/projects/7f7b14c04c694733bb036c5c369b023a/users/ece311c6ff38448b9be3ede7c6dafb0a/roles/56e4a96094e9434cbbafb22579c034f5 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 22:29:15.556180 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec31f34-4083-402e-8c1d-2c04fb297c2f None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:15.556507 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec31f34-4083-402e-8c1d-2c04fb297c2f None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:15.556791 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec31f34-4083-402e-8c1d-2c04fb297c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:15.584514 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-4ec31f34-4083-402e-8c1d-2c04fb297c2f None None] MFA Rules not processed for user `ece311c6ff38448b9be3ede7c6dafb0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:29:15.585733 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-4ec31f34-4083-402e-8c1d-2c04fb297c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:29:15.649168 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1406/2812] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:29:15 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:29:15.653640 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a8d46d-8798-4d90-bad1-720fd8e05a9e admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:15.716645 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a8d46d-8798-4d90-bad1-720fd8e05a9e admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:15.721563 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a0d804-f660-458b-9139-587d75b2f4c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:15.722161 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a0d804-f660-458b-9139-587d75b2f4c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:15.722443 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a0d804-f660-458b-9139-587d75b2f4c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:15.722718 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a0d804-f660-458b-9139-587d75b2f4c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:15.723337 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a0d804-f660-458b-9139-587d75b2f4c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:15.726195 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a0d804-f660-458b-9139-587d75b2f4c4 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:15.740216 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:29:15.814139 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1407/2813] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:29:15 2022] POST /identity/v3/projects => generated 369 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 22:29:15.818747 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec31f34-4083-402e-8c1d-2c04fb297c2f None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:15.829507 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec31f34-4083-402e-8c1d-2c04fb297c2f None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:15.871099 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:29:15.887613 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af20c28-3699-4633-98cd-5d589d2f50c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:15.888238 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af20c28-3699-4633-98cd-5d589d2f50c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:15.888545 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af20c28-3699-4633-98cd-5d589d2f50c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:15.888816 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af20c28-3699-4633-98cd-5d589d2f50c7 admin admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:15.889446 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af20c28-3699-4633-98cd-5d589d2f50c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:15.891390 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af20c28-3699-4633-98cd-5d589d2f50c7 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:15.979199 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1407/2814] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:29:15 2022] POST /identity/v3/users => generated 291 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:29:15.983075 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a0d804-f660-458b-9139-587d75b2f4c4 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:15.993436 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a0d804-f660-458b-9139-587d75b2f4c4 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:15.998039 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d1e8f7-b79d-4522-a429-687c75a92b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:15.998625 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d1e8f7-b79d-4522-a429-687c75a92b63 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:15.998899 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d1e8f7-b79d-4522-a429-687c75a92b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:15.999168 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d1e8f7-b79d-4522-a429-687c75a92b63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:15.999796 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d1e8f7-b79d-4522-a429-687c75a92b63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:16.001484 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d1e8f7-b79d-4522-a429-687c75a92b63 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:16.012587 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1408/2815] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:29:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:29:16.016536 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af20c28-3699-4633-98cd-5d589d2f50c7 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:16.027502 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af20c28-3699-4633-98cd-5d589d2f50c7 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:16.032229 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3fa254-3b54-4a63-bfb1-2d4068f2803a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:16.032911 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3fa254-3b54-4a63-bfb1-2d4068f2803a admin admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:16.033193 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3fa254-3b54-4a63-bfb1-2d4068f2803a admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:16.033469 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3fa254-3b54-4a63-bfb1-2d4068f2803a admin admin] PATH_INFO: `/v3/projects/d59aa9238e04447f8b7ae917e3ef9f5b/users/9a2ff060802b48ef8a5ce93444cdf0b4/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:16.048268 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3fa254-3b54-4a63-bfb1-2d4068f2803a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d59aa9238e04447f8b7ae917e3ef9f5b, user_id=9a2ff060802b48ef8a5ce93444cdf0b4, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:16.050993 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3fa254-3b54-4a63-bfb1-2d4068f2803a admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:16.128773 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1408/2816] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:29:16 2022] PUT /identity/v3/projects/d59aa9238e04447f8b7ae917e3ef9f5b/users/9a2ff060802b48ef8a5ce93444cdf0b4/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:29:16.133308 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d1e8f7-b79d-4522-a429-687c75a92b63 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:16.143179 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d1e8f7-b79d-4522-a429-687c75a92b63 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:16.199442 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c603d84-a3bd-4d2a-a1c4-1ec157ca65c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:16.200031 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c603d84-a3bd-4d2a-a1c4-1ec157ca65c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:16.200301 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c603d84-a3bd-4d2a-a1c4-1ec157ca65c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:16.200602 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c603d84-a3bd-4d2a-a1c4-1ec157ca65c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:16.201225 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c603d84-a3bd-4d2a-a1c4-1ec157ca65c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:16.202808 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c603d84-a3bd-4d2a-a1c4-1ec157ca65c6 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:16.212585 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1409/2817] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:29:16 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 22:29:16.216704 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3fa254-3b54-4a63-bfb1-2d4068f2803a admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:16.227150 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3fa254-3b54-4a63-bfb1-2d4068f2803a admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:16.231745 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7096a41-07e2-4442-b22e-f47cd63d26bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:16.232353 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7096a41-07e2-4442-b22e-f47cd63d26bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:16.232662 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7096a41-07e2-4442-b22e-f47cd63d26bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:16.232932 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7096a41-07e2-4442-b22e-f47cd63d26bf admin admin] PATH_INFO: `/v3/projects/d59aa9238e04447f8b7ae917e3ef9f5b/users/9a2ff060802b48ef8a5ce93444cdf0b4/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:16.235169 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7096a41-07e2-4442-b22e-f47cd63d26bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=d59aa9238e04447f8b7ae917e3ef9f5b, user_id=9a2ff060802b48ef8a5ce93444cdf0b4, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:16.237819 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7096a41-07e2-4442-b22e-f47cd63d26bf admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:16.248167 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1409/2818] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:29:16 2022] PUT /identity/v3/projects/d59aa9238e04447f8b7ae917e3ef9f5b/users/9a2ff060802b48ef8a5ce93444cdf0b4/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:29:16.258725 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deed5f48-d5f3-4107-92e3-ba9347f8b202 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:16.259004 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deed5f48-d5f3-4107-92e3-ba9347f8b202 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:16.259300 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deed5f48-d5f3-4107-92e3-ba9347f8b202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:16.285809 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-deed5f48-d5f3-4107-92e3-ba9347f8b202 None None] MFA Rules not processed for user `9a2ff060802b48ef8a5ce93444cdf0b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:29:16.287327 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-deed5f48-d5f3-4107-92e3-ba9347f8b202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:29:16.353516 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1410/2819] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:29:16 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:29:16.365838 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7096a41-07e2-4442-b22e-f47cd63d26bf admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:16.367605 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-b7096a41-07e2-4442-b22e-f47cd63d26bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:29:16.424836 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7096a41-07e2-4442-b22e-f47cd63d26bf admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:16.429079 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3723f0e7-40de-4307-92ed-479918cbe4e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa71c71de263478da234f3c5716a6e5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:16.429677 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3723f0e7-40de-4307-92ed-479918cbe4e4 service cinder] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:16.429971 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3723f0e7-40de-4307-92ed-479918cbe4e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:16.430223 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3723f0e7-40de-4307-92ed-479918cbe4e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:16.439277 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3723f0e7-40de-4307-92ed-479918cbe4e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:16.440730 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3723f0e7-40de-4307-92ed-479918cbe4e4 service cinder] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:16.450764 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1410/2820] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:29:16 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:29:16.561550 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deed5f48-d5f3-4107-92e3-ba9347f8b202 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:16.562989 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-deed5f48-d5f3-4107-92e3-ba9347f8b202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:29:16.575546 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deed5f48-d5f3-4107-92e3-ba9347f8b202 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:16.579261 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d02751-11f0-4637-9432-334948bbe348 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa71c71de263478da234f3c5716a6e5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:16.579882 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d02751-11f0-4637-9432-334948bbe348 service cinder] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:16.580165 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d02751-11f0-4637-9432-334948bbe348 service cinder] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:16.580478 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d02751-11f0-4637-9432-334948bbe348 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:16.638270 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d02751-11f0-4637-9432-334948bbe348 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:16.639700 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d02751-11f0-4637-9432-334948bbe348 service cinder] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:16.648897 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1411/2821] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:29:16 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:29:21.583699 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3723f0e7-40de-4307-92ed-479918cbe4e4 service cinder] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:21.645424 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3723f0e7-40de-4307-92ed-479918cbe4e4 service cinder] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:21.650568 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9315943f-1b09-470e-8894-86d1e603cd03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:21.651186 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9315943f-1b09-470e-8894-86d1e603cd03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:21.652624 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9315943f-1b09-470e-8894-86d1e603cd03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:21.652624 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9315943f-1b09-470e-8894-86d1e603cd03 admin admin] PATH_INFO: `/v3/users/ece311c6ff38448b9be3ede7c6dafb0a` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:21.652847 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9315943f-1b09-470e-8894-86d1e603cd03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ece311c6ff38448b9be3ede7c6dafb0a)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:21.654789 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9315943f-1b09-470e-8894-86d1e603cd03 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:21.805854 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:29:21.811698 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-9315943f-1b09-470e-8894-86d1e603cd03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ece311c6ff38448b9be3ede7c6dafb0a'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:21.823965 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-9315943f-1b09-470e-8894-86d1e603cd03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ece311c6ff38448b9be3ede7c6dafb0a'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:21.895211 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-9315943f-1b09-470e-8894-86d1e603cd03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ece311c6ff38448b9be3ede7c6dafb0a'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:21.903405 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-9315943f-1b09-470e-8894-86d1e603cd03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ece311c6ff38448b9be3ede7c6dafb0a'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:21.904069 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-9315943f-1b09-470e-8894-86d1e603cd03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ece311c6ff38448b9be3ede7c6dafb0a'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:21.910231 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1411/2822] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:29:21 2022] DELETE /identity/v3/users/ece311c6ff38448b9be3ede7c6dafb0a => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 22:29:21.914767 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5cdf5-6ae3-4d11-80b9-4dd5b354bc40 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:21.915265 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5cdf5-6ae3-4d11-80b9-4dd5b354bc40 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:21.915371 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5cdf5-6ae3-4d11-80b9-4dd5b354bc40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:21.922464 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:29:21.934864 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-3fb5cdf5-6ae3-4d11-80b9-4dd5b354bc40 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:29:21.936071 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-3fb5cdf5-6ae3-4d11-80b9-4dd5b354bc40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:29:21.993538 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1412/2823] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:29:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:29:22.003373 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9315943f-1b09-470e-8894-86d1e603cd03 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:22.004793 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-9315943f-1b09-470e-8894-86d1e603cd03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:29:22.062329 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9315943f-1b09-470e-8894-86d1e603cd03 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:22.066024 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3decc97-1ca9-4dff-bb25-debdcfa0e4a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:22.066618 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3decc97-1ca9-4dff-bb25-debdcfa0e4a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:22.066891 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3decc97-1ca9-4dff-bb25-debdcfa0e4a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:22.067156 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3decc97-1ca9-4dff-bb25-debdcfa0e4a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:22.076691 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3decc97-1ca9-4dff-bb25-debdcfa0e4a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:22.078039 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3decc97-1ca9-4dff-bb25-debdcfa0e4a4 service neutron] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:22.088142 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1412/2824] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:29:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:29:22.542808 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb5cdf5-6ae3-4d11-80b9-4dd5b354bc40 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:22.544330 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-3fb5cdf5-6ae3-4d11-80b9-4dd5b354bc40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:29:22.557385 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb5cdf5-6ae3-4d11-80b9-4dd5b354bc40 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:22.561922 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260d6aba-dc1e-4203-a2e9-a6a508e15f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:22.562591 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260d6aba-dc1e-4203-a2e9-a6a508e15f33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:22.562863 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260d6aba-dc1e-4203-a2e9-a6a508e15f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:22.563129 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260d6aba-dc1e-4203-a2e9-a6a508e15f33 admin admin] PATH_INFO: `/v3/projects/7f7b14c04c694733bb036c5c369b023a` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:22.564232 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260d6aba-dc1e-4203-a2e9-a6a508e15f33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f7b14c04c694733bb036c5c369b023a)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:22.566293 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260d6aba-dc1e-4203-a2e9-a6a508e15f33 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:22.679730 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-260d6aba-dc1e-4203-a2e9-a6a508e15f33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f7b14c04c694733bb036c5c369b023a', 'request_id': 'req-260d6aba-dc1e-4203-a2e9-a6a508e15f33'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:22.773042 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-260d6aba-dc1e-4203-a2e9-a6a508e15f33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f7b14c04c694733bb036c5c369b023a', 'request_id': 'req-260d6aba-dc1e-4203-a2e9-a6a508e15f33'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:22.779170 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-260d6aba-dc1e-4203-a2e9-a6a508e15f33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f7b14c04c694733bb036c5c369b023a', 'request_id': 'req-260d6aba-dc1e-4203-a2e9-a6a508e15f33'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:22.790983 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-260d6aba-dc1e-4203-a2e9-a6a508e15f33 admin admin] The token cache is being invalidate because project 7f7b14c04c694733bb036c5c369b023a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:29:22.791268 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-260d6aba-dc1e-4203-a2e9-a6a508e15f33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:22.791894 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-260d6aba-dc1e-4203-a2e9-a6a508e15f33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:22.793395 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1413/2825] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:29:22 2022] DELETE /identity/v3/projects/7f7b14c04c694733bb036c5c369b023a => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:29:22.797367 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3decc97-1ca9-4dff-bb25-debdcfa0e4a4 service neutron] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:22.807978 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3decc97-1ca9-4dff-bb25-debdcfa0e4a4 service neutron] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:22.863703 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f585c44-4972-41c9-96fe-978e8980c6ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:22.864324 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f585c44-4972-41c9-96fe-978e8980c6ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:22.864631 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f585c44-4972-41c9-96fe-978e8980c6ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:22.864906 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f585c44-4972-41c9-96fe-978e8980c6ca admin admin] PATH_INFO: `/v3/users/9a2ff060802b48ef8a5ce93444cdf0b4` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:22.866770 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f585c44-4972-41c9-96fe-978e8980c6ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a2ff060802b48ef8a5ce93444cdf0b4)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:22.868751 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f585c44-4972-41c9-96fe-978e8980c6ca admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:23.051351 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-1f585c44-4972-41c9-96fe-978e8980c6ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a2ff060802b48ef8a5ce93444cdf0b4'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:23.061445 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:29:23.063638 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-1f585c44-4972-41c9-96fe-978e8980c6ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a2ff060802b48ef8a5ce93444cdf0b4'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:23.121830 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-1f585c44-4972-41c9-96fe-978e8980c6ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a2ff060802b48ef8a5ce93444cdf0b4'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:23.128992 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-1f585c44-4972-41c9-96fe-978e8980c6ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a2ff060802b48ef8a5ce93444cdf0b4'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:23.129910 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-1f585c44-4972-41c9-96fe-978e8980c6ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a2ff060802b48ef8a5ce93444cdf0b4'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:23.135965 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1413/2826] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:29:22 2022] DELETE /identity/v3/users/9a2ff060802b48ef8a5ce93444cdf0b4 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:29:23.575005 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260d6aba-dc1e-4203-a2e9-a6a508e15f33 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:23.576611 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-260d6aba-dc1e-4203-a2e9-a6a508e15f33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:29:23.594083 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260d6aba-dc1e-4203-a2e9-a6a508e15f33 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:23.651034 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4906c4c-945c-4c7e-9708-fb2c2251566b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:23.651661 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4906c4c-945c-4c7e-9708-fb2c2251566b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:23.651943 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4906c4c-945c-4c7e-9708-fb2c2251566b admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:23.652215 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4906c4c-945c-4c7e-9708-fb2c2251566b admin admin] PATH_INFO: `/v3/projects/d59aa9238e04447f8b7ae917e3ef9f5b` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:23.653356 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4906c4c-945c-4c7e-9708-fb2c2251566b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d59aa9238e04447f8b7ae917e3ef9f5b)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:23.655262 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4906c4c-945c-4c7e-9708-fb2c2251566b admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:23.769800 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:29:23.784098 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-e4906c4c-945c-4c7e-9708-fb2c2251566b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd59aa9238e04447f8b7ae917e3ef9f5b', 'request_id': 'req-e4906c4c-945c-4c7e-9708-fb2c2251566b'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:23.869929 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-e4906c4c-945c-4c7e-9708-fb2c2251566b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd59aa9238e04447f8b7ae917e3ef9f5b', 'request_id': 'req-e4906c4c-945c-4c7e-9708-fb2c2251566b'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:23.875392 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-e4906c4c-945c-4c7e-9708-fb2c2251566b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd59aa9238e04447f8b7ae917e3ef9f5b', 'request_id': 'req-e4906c4c-945c-4c7e-9708-fb2c2251566b'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:23.888088 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-e4906c4c-945c-4c7e-9708-fb2c2251566b admin admin] The token cache is being invalidate because project d59aa9238e04447f8b7ae917e3ef9f5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:29:23.888380 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-e4906c4c-945c-4c7e-9708-fb2c2251566b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:23.889041 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-e4906c4c-945c-4c7e-9708-fb2c2251566b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:23.890586 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1414/2827] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:29:23 2022] DELETE /identity/v3/projects/d59aa9238e04447f8b7ae917e3ef9f5b => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:29:23.895682 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff91d7d-ae14-4672-9cda-d0f957a5d376 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:23.898815 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff91d7d-ae14-4672-9cda-d0f957a5d376 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:23.898815 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff91d7d-ae14-4672-9cda-d0f957a5d376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:23.915697 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-1ff91d7d-ae14-4672-9cda-d0f957a5d376 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:29:23.916928 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-1ff91d7d-ae14-4672-9cda-d0f957a5d376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:29:23.976635 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1414/2828] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:29:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:29:23.984652 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f505a68-bb3b-45bb-b0c3-ed92b4d3eb38 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:23.984954 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f505a68-bb3b-45bb-b0c3-ed92b4d3eb38 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:23.985238 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f505a68-bb3b-45bb-b0c3-ed92b4d3eb38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:24.005200 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-9f505a68-bb3b-45bb-b0c3-ed92b4d3eb38 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:29:24.006425 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-9f505a68-bb3b-45bb-b0c3-ed92b4d3eb38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:29:24.014391 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1415/2829] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:29:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:29:24.018450 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff91d7d-ae14-4672-9cda-d0f957a5d376 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:24.028100 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff91d7d-ae14-4672-9cda-d0f957a5d376 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:24.032895 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9483dc5f-ba3e-4764-be72-47d5b172eed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:24.033499 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9483dc5f-ba3e-4764-be72-47d5b172eed1 admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:24.033798 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9483dc5f-ba3e-4764-be72-47d5b172eed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:24.034069 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9483dc5f-ba3e-4764-be72-47d5b172eed1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:24.034715 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9483dc5f-ba3e-4764-be72-47d5b172eed1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 22:29:24.035016 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9483dc5f-ba3e-4764-be72-47d5b172eed1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:24.036666 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9483dc5f-ba3e-4764-be72-47d5b172eed1 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:24.048800 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1415/2830] 10.222.0.18 () {58 vars in 1153 bytes} [Sat Aug 6 22:29:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 22:29:24.052831 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f505a68-bb3b-45bb-b0c3-ed92b4d3eb38 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:24.062481 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f505a68-bb3b-45bb-b0c3-ed92b4d3eb38 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:24.066983 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6e2804-016c-4d71-89d4-e0a7f2a72d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:24.067562 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6e2804-016c-4d71-89d4-e0a7f2a72d58 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:24.067837 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6e2804-016c-4d71-89d4-e0a7f2a72d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:24.068107 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6e2804-016c-4d71-89d4-e0a7f2a72d58 admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:24.068759 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6e2804-016c-4d71-89d4-e0a7f2a72d58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:24.070641 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6e2804-016c-4d71-89d4-e0a7f2a72d58 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:24.180649 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1416/2831] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:29:24 2022] POST /identity/v3/projects => generated 383 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:29:24.184813 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9483dc5f-ba3e-4764-be72-47d5b172eed1 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:24.195009 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9483dc5f-ba3e-4764-be72-47d5b172eed1 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:24.250759 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72f51a8-afe2-497a-863b-8432180be43c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:24.251363 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72f51a8-afe2-497a-863b-8432180be43c admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:24.251643 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72f51a8-afe2-497a-863b-8432180be43c admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:24.251924 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72f51a8-afe2-497a-863b-8432180be43c admin admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:24.252621 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72f51a8-afe2-497a-863b-8432180be43c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:24.254572 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72f51a8-afe2-497a-863b-8432180be43c admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:24.346336 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1416/2832] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:29:24 2022] POST /identity/v3/users => generated 292 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:29:24.351218 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6e2804-016c-4d71-89d4-e0a7f2a72d58 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:24.362051 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6e2804-016c-4d71-89d4-e0a7f2a72d58 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:24.367323 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a256579-7f62-4c56-8948-37864dd6aa4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:24.367921 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a256579-7f62-4c56-8948-37864dd6aa4a admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:24.368194 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a256579-7f62-4c56-8948-37864dd6aa4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:24.368503 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a256579-7f62-4c56-8948-37864dd6aa4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:24.369140 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a256579-7f62-4c56-8948-37864dd6aa4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:24.370727 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a256579-7f62-4c56-8948-37864dd6aa4a admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:24.380892 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1417/2833] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:29:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:29:24.384792 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72f51a8-afe2-497a-863b-8432180be43c admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:24.394586 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72f51a8-afe2-497a-863b-8432180be43c admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:24.399849 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9697b17-2bbc-4298-a885-da85adfc3539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:24.400463 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9697b17-2bbc-4298-a885-da85adfc3539 admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:24.400741 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9697b17-2bbc-4298-a885-da85adfc3539 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:24.401008 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9697b17-2bbc-4298-a885-da85adfc3539 admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:24.401674 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9697b17-2bbc-4298-a885-da85adfc3539 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:24.403249 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9697b17-2bbc-4298-a885-da85adfc3539 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:24.415342 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1417/2834] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:29:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:29:24.419384 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a256579-7f62-4c56-8948-37864dd6aa4a admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:24.429399 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a256579-7f62-4c56-8948-37864dd6aa4a admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:24.434000 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f60305-c44b-4afa-804f-f5bb307fb2e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:24.434579 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f60305-c44b-4afa-804f-f5bb307fb2e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:24.434860 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f60305-c44b-4afa-804f-f5bb307fb2e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:24.435135 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f60305-c44b-4afa-804f-f5bb307fb2e6 admin admin] PATH_INFO: `/v3/projects/aa3edc73065e4775905ec44c75f5788b/users/8ae1ba9630414903bd36fadc51f89790/roles/56e4a96094e9434cbbafb22579c034f5` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:24.449628 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f60305-c44b-4afa-804f-f5bb307fb2e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa3edc73065e4775905ec44c75f5788b, user_id=8ae1ba9630414903bd36fadc51f89790, role_id=56e4a96094e9434cbbafb22579c034f5)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:24.452303 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f60305-c44b-4afa-804f-f5bb307fb2e6 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:24.511822 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1418/2835] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:29:24 2022] PUT /identity/v3/projects/aa3edc73065e4775905ec44c75f5788b/users/8ae1ba9630414903bd36fadc51f89790/roles/56e4a96094e9434cbbafb22579c034f5 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:29:24.577371 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f541fe-89ec-45ca-ac71-3dd92add1042 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:24.577678 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f541fe-89ec-45ca-ac71-3dd92add1042 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:24.577950 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f541fe-89ec-45ca-ac71-3dd92add1042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:24.606087 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-33f541fe-89ec-45ca-ac71-3dd92add1042 None None] MFA Rules not processed for user `8ae1ba9630414903bd36fadc51f89790`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:29:24.607273 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-33f541fe-89ec-45ca-ac71-3dd92add1042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:29:24.638614 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:29:24.670918 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1418/2836] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:29:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:29:24.675579 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f60305-c44b-4afa-804f-f5bb307fb2e6 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:24.739333 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f60305-c44b-4afa-804f-f5bb307fb2e6 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:24.743969 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed77ab58-0719-4dbb-85b8-ff966f00836b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:24.744595 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed77ab58-0719-4dbb-85b8-ff966f00836b admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:24.744872 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed77ab58-0719-4dbb-85b8-ff966f00836b admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:24.745136 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed77ab58-0719-4dbb-85b8-ff966f00836b admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:24.745749 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed77ab58-0719-4dbb-85b8-ff966f00836b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:24.747619 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed77ab58-0719-4dbb-85b8-ff966f00836b admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:24.827254 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1419/2837] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:29:24 2022] POST /identity/v3/projects => generated 385 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:29:24.831399 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f541fe-89ec-45ca-ac71-3dd92add1042 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:24.843126 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f541fe-89ec-45ca-ac71-3dd92add1042 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:24.898935 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e10965-24ec-41f5-95e8-56f178997f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:24.899595 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e10965-24ec-41f5-95e8-56f178997f0d admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:24.899793 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e10965-24ec-41f5-95e8-56f178997f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:24.900062 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e10965-24ec-41f5-95e8-56f178997f0d admin admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:24.900742 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e10965-24ec-41f5-95e8-56f178997f0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:24.902642 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e10965-24ec-41f5-95e8-56f178997f0d admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:24.969124 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1419/2838] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:29:24 2022] POST /identity/v3/users => generated 293 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:29:24.973393 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed77ab58-0719-4dbb-85b8-ff966f00836b admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:24.983603 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed77ab58-0719-4dbb-85b8-ff966f00836b admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:24.988295 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01012583-8986-46d1-aabd-f91e989e3ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:24.988987 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01012583-8986-46d1-aabd-f91e989e3ef0 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:24.989267 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01012583-8986-46d1-aabd-f91e989e3ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:24.989588 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01012583-8986-46d1-aabd-f91e989e3ef0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:24.990175 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01012583-8986-46d1-aabd-f91e989e3ef0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:24.991848 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01012583-8986-46d1-aabd-f91e989e3ef0 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:25.002230 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1420/2839] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:29:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:29:25.006593 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e10965-24ec-41f5-95e8-56f178997f0d admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:25.016613 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e10965-24ec-41f5-95e8-56f178997f0d admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:25.021284 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0306248a-b756-4bff-bcc8-78fbdc3e6903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:25.022506 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0306248a-b756-4bff-bcc8-78fbdc3e6903 admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:25.022781 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0306248a-b756-4bff-bcc8-78fbdc3e6903 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:25.023046 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0306248a-b756-4bff-bcc8-78fbdc3e6903 admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:25.023725 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0306248a-b756-4bff-bcc8-78fbdc3e6903 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:25.025370 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0306248a-b756-4bff-bcc8-78fbdc3e6903 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:25.035281 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1420/2840] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:29:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:29:25.039246 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01012583-8986-46d1-aabd-f91e989e3ef0 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:25.048926 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01012583-8986-46d1-aabd-f91e989e3ef0 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:25.053585 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912343c0-b954-44f6-9df3-06584afe040f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:25.054217 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912343c0-b954-44f6-9df3-06584afe040f admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:25.054494 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912343c0-b954-44f6-9df3-06584afe040f admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:25.054764 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912343c0-b954-44f6-9df3-06584afe040f admin admin] PATH_INFO: `/v3/projects/8151ae7deed644e0b83338678586f64a/users/559ff62257b84d589750bb9277a34b45/roles/56e4a96094e9434cbbafb22579c034f5` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:25.069271 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912343c0-b954-44f6-9df3-06584afe040f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8151ae7deed644e0b83338678586f64a, user_id=559ff62257b84d589750bb9277a34b45, role_id=56e4a96094e9434cbbafb22579c034f5)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:25.071902 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912343c0-b954-44f6-9df3-06584afe040f admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:25.145829 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1421/2841] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:29:25 2022] PUT /identity/v3/projects/8151ae7deed644e0b83338678586f64a/users/559ff62257b84d589750bb9277a34b45/roles/56e4a96094e9434cbbafb22579c034f5 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 22:29:25.155842 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3874047-ba34-4f16-838a-79333d6dbf40 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:25.156130 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3874047-ba34-4f16-838a-79333d6dbf40 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:25.156496 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3874047-ba34-4f16-838a-79333d6dbf40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:25.185272 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-e3874047-ba34-4f16-838a-79333d6dbf40 None None] MFA Rules not processed for user `559ff62257b84d589750bb9277a34b45`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:29:25.186505 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-e3874047-ba34-4f16-838a-79333d6dbf40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:29:25.250122 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1421/2842] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:29:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:29:25.254497 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912343c0-b954-44f6-9df3-06584afe040f admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:25.259450 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:29:25.314970 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912343c0-b954-44f6-9df3-06584afe040f admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:25.320683 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2550a1-0761-44a0-8980-320b5b135324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:25.321274 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2550a1-0761-44a0-8980-320b5b135324 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:25.321547 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2550a1-0761-44a0-8980-320b5b135324 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:25.321838 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2550a1-0761-44a0-8980-320b5b135324 admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:25.322453 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2550a1-0761-44a0-8980-320b5b135324 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:25.324372 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2550a1-0761-44a0-8980-320b5b135324 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:25.399561 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1422/2843] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:29:25 2022] POST /identity/v3/projects => generated 387 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 22:29:25.403695 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3874047-ba34-4f16-838a-79333d6dbf40 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:25.414538 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3874047-ba34-4f16-838a-79333d6dbf40 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:25.454510 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:29:25.471000 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b6aa77-1b6d-44fa-806e-b20819fad665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:25.471542 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b6aa77-1b6d-44fa-806e-b20819fad665 admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:25.471816 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b6aa77-1b6d-44fa-806e-b20819fad665 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:25.472082 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b6aa77-1b6d-44fa-806e-b20819fad665 admin admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:25.472741 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b6aa77-1b6d-44fa-806e-b20819fad665 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:25.474634 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b6aa77-1b6d-44fa-806e-b20819fad665 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:25.563003 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1422/2844] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:29:25 2022] POST /identity/v3/users => generated 300 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:29:25.566865 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2550a1-0761-44a0-8980-320b5b135324 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:25.577270 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2550a1-0761-44a0-8980-320b5b135324 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:25.581880 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19584ad-41ae-45d6-9397-b7d6de5028df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:25.582474 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19584ad-41ae-45d6-9397-b7d6de5028df admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:25.582748 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19584ad-41ae-45d6-9397-b7d6de5028df admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:25.583044 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19584ad-41ae-45d6-9397-b7d6de5028df admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:25.583675 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19584ad-41ae-45d6-9397-b7d6de5028df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:25.585625 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19584ad-41ae-45d6-9397-b7d6de5028df admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:25.596384 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1423/2845] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:29:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:29:25.601045 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b6aa77-1b6d-44fa-806e-b20819fad665 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:25.610846 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b6aa77-1b6d-44fa-806e-b20819fad665 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:25.616047 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cead85f-e9d8-4596-8998-f948ef8d6a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:25.616728 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cead85f-e9d8-4596-8998-f948ef8d6a91 admin admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:25.617007 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cead85f-e9d8-4596-8998-f948ef8d6a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:25.617278 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cead85f-e9d8-4596-8998-f948ef8d6a91 admin admin] PATH_INFO: `/v3/projects/a4cf4f56340d4c029d8b33bd5eff6466/users/fcbcf62fb69a49648ae7023b4eef2585/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:25.631800 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cead85f-e9d8-4596-8998-f948ef8d6a91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4cf4f56340d4c029d8b33bd5eff6466, user_id=fcbcf62fb69a49648ae7023b4eef2585, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:25.634458 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cead85f-e9d8-4596-8998-f948ef8d6a91 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:25.689610 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1423/2846] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:29:25 2022] PUT /identity/v3/projects/a4cf4f56340d4c029d8b33bd5eff6466/users/fcbcf62fb69a49648ae7023b4eef2585/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:29:25.693723 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19584ad-41ae-45d6-9397-b7d6de5028df admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:25.703868 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19584ad-41ae-45d6-9397-b7d6de5028df admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:25.759779 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f372567-af3b-49ae-8bda-2d65223e2fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:25.760375 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f372567-af3b-49ae-8bda-2d65223e2fa9 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:25.760682 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f372567-af3b-49ae-8bda-2d65223e2fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:25.760953 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f372567-af3b-49ae-8bda-2d65223e2fa9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:25.761617 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f372567-af3b-49ae-8bda-2d65223e2fa9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:25.763244 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f372567-af3b-49ae-8bda-2d65223e2fa9 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:25.772845 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1424/2847] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:29:25 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:29:25.777015 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cead85f-e9d8-4596-8998-f948ef8d6a91 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:25.787859 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cead85f-e9d8-4596-8998-f948ef8d6a91 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:25.792668 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b03b030-3b0e-46af-9bd2-a2507fd4494b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:25.793345 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b03b030-3b0e-46af-9bd2-a2507fd4494b admin admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:25.793625 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b03b030-3b0e-46af-9bd2-a2507fd4494b admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:25.793897 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b03b030-3b0e-46af-9bd2-a2507fd4494b admin admin] PATH_INFO: `/v3/projects/a4cf4f56340d4c029d8b33bd5eff6466/users/fcbcf62fb69a49648ae7023b4eef2585/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:25.795838 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b03b030-3b0e-46af-9bd2-a2507fd4494b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4cf4f56340d4c029d8b33bd5eff6466, user_id=fcbcf62fb69a49648ae7023b4eef2585, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:25.798472 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b03b030-3b0e-46af-9bd2-a2507fd4494b admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:25.809165 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1424/2848] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:29:25 2022] PUT /identity/v3/projects/a4cf4f56340d4c029d8b33bd5eff6466/users/fcbcf62fb69a49648ae7023b4eef2585/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:29:25.820421 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35fddfb-2be6-4c92-97b2-ca49c890e1b2 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:25.820648 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35fddfb-2be6-4c92-97b2-ca49c890e1b2 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:25.820925 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35fddfb-2be6-4c92-97b2-ca49c890e1b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:25.848105 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-e35fddfb-2be6-4c92-97b2-ca49c890e1b2 None None] MFA Rules not processed for user `fcbcf62fb69a49648ae7023b4eef2585`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:29:25.849350 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-e35fddfb-2be6-4c92-97b2-ca49c890e1b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:29:25.913328 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:29:25.918441 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1425/2849] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:29:25 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 22:29:25.929440 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b03b030-3b0e-46af-9bd2-a2507fd4494b admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:25.931324 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-9b03b030-3b0e-46af-9bd2-a2507fd4494b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:29:25.995579 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b03b030-3b0e-46af-9bd2-a2507fd4494b admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:25.999496 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d95fc0b-43e5-4753-8fb1-f6236764213d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa71c71de263478da234f3c5716a6e5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:26.000091 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d95fc0b-43e5-4753-8fb1-f6236764213d service cinder] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:26.000376 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d95fc0b-43e5-4753-8fb1-f6236764213d service cinder] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:26.000682 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d95fc0b-43e5-4753-8fb1-f6236764213d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:26.009922 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d95fc0b-43e5-4753-8fb1-f6236764213d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:26.011346 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d95fc0b-43e5-4753-8fb1-f6236764213d service cinder] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:26.020976 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1425/2850] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:29:25 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:29:27.119293 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35fddfb-2be6-4c92-97b2-ca49c890e1b2 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:27.120745 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-e35fddfb-2be6-4c92-97b2-ca49c890e1b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:29:27.132301 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35fddfb-2be6-4c92-97b2-ca49c890e1b2 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:27.135748 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c46eb60-325f-4ca4-831d-14a01e48be8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa71c71de263478da234f3c5716a6e5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:27.136328 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c46eb60-325f-4ca4-831d-14a01e48be8f service cinder] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:27.136639 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c46eb60-325f-4ca4-831d-14a01e48be8f service cinder] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:27.136913 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c46eb60-325f-4ca4-831d-14a01e48be8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:27.191300 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c46eb60-325f-4ca4-831d-14a01e48be8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:27.192760 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c46eb60-325f-4ca4-831d-14a01e48be8f service cinder] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:27.201950 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1426/2851] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:29:27 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:29:33.179203 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d95fc0b-43e5-4753-8fb1-f6236764213d service cinder] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:33.187843 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d95fc0b-43e5-4753-8fb1-f6236764213d service cinder] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:33.191464 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c3fc32-3eee-4a43-b635-a267fcf5fec7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa71c71de263478da234f3c5716a6e5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:33.192050 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c3fc32-3eee-4a43-b635-a267fcf5fec7 service cinder] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:33.192322 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c3fc32-3eee-4a43-b635-a267fcf5fec7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:33.192618 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c3fc32-3eee-4a43-b635-a267fcf5fec7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:33.248890 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c3fc32-3eee-4a43-b635-a267fcf5fec7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:33.250312 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c3fc32-3eee-4a43-b635-a267fcf5fec7 service cinder] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:33.259392 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1426/2852] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:29:33 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:29:37.098468 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c46eb60-325f-4ca4-831d-14a01e48be8f service cinder] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:37.160281 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c46eb60-325f-4ca4-831d-14a01e48be8f service cinder] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:37.164991 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c028a9-e48c-4701-af49-f92ea506bd82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:37.165607 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c028a9-e48c-4701-af49-f92ea506bd82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:37.165886 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c028a9-e48c-4701-af49-f92ea506bd82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:37.166160 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c028a9-e48c-4701-af49-f92ea506bd82 admin admin] PATH_INFO: `/v3/users/8ae1ba9630414903bd36fadc51f89790` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:37.167365 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c028a9-e48c-4701-af49-f92ea506bd82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ae1ba9630414903bd36fadc51f89790)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:37.169319 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c028a9-e48c-4701-af49-f92ea506bd82 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:37.383352 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-66c028a9-e48c-4701-af49-f92ea506bd82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ae1ba9630414903bd36fadc51f89790'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:37.395254 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-66c028a9-e48c-4701-af49-f92ea506bd82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ae1ba9630414903bd36fadc51f89790'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:37.402284 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-66c028a9-e48c-4701-af49-f92ea506bd82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ae1ba9630414903bd36fadc51f89790'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:37.402933 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-66c028a9-e48c-4701-af49-f92ea506bd82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ae1ba9630414903bd36fadc51f89790'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:37.403521 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-66c028a9-e48c-4701-af49-f92ea506bd82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ae1ba9630414903bd36fadc51f89790'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:37.488581 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1427/2853] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:29:37 2022] DELETE /identity/v3/users/8ae1ba9630414903bd36fadc51f89790 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:29:37.493203 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23308f3f-d4e4-4297-a035-91c8f9ba3ef9 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:37.493736 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23308f3f-d4e4-4297-a035-91c8f9ba3ef9 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:37.493880 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23308f3f-d4e4-4297-a035-91c8f9ba3ef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:37.512923 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-23308f3f-d4e4-4297-a035-91c8f9ba3ef9 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:29:37.514110 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-23308f3f-d4e4-4297-a035-91c8f9ba3ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:29:37.545747 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:29:37.575058 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1427/2854] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:29:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 22:29:37.585632 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c028a9-e48c-4701-af49-f92ea506bd82 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:37.587028 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-66c028a9-e48c-4701-af49-f92ea506bd82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:29:37.605666 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:29:37.643991 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c028a9-e48c-4701-af49-f92ea506bd82 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:37.648390 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf239252-98a0-4a10-b9cf-90f9878c4ad2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:37.649234 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf239252-98a0-4a10-b9cf-90f9878c4ad2 service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:37.649554 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf239252-98a0-4a10-b9cf-90f9878c4ad2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:37.649842 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf239252-98a0-4a10-b9cf-90f9878c4ad2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:37.659154 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf239252-98a0-4a10-b9cf-90f9878c4ad2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:37.660619 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf239252-98a0-4a10-b9cf-90f9878c4ad2 service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:37.670093 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1428/2855] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:29:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:29:38.065616 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23308f3f-d4e4-4297-a035-91c8f9ba3ef9 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:38.067930 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-23308f3f-d4e4-4297-a035-91c8f9ba3ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:29:38.080943 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23308f3f-d4e4-4297-a035-91c8f9ba3ef9 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:38.086238 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4456f2-511c-4e40-b9e0-ca65f19c4157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:38.086868 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4456f2-511c-4e40-b9e0-ca65f19c4157 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:38.087144 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4456f2-511c-4e40-b9e0-ca65f19c4157 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:38.087982 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4456f2-511c-4e40-b9e0-ca65f19c4157 admin admin] PATH_INFO: `/v3/projects/aa3edc73065e4775905ec44c75f5788b` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:38.089349 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4456f2-511c-4e40-b9e0-ca65f19c4157 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa3edc73065e4775905ec44c75f5788b)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:38.091307 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4456f2-511c-4e40-b9e0-ca65f19c4157 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:38.195750 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-9f4456f2-511c-4e40-b9e0-ca65f19c4157 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa3edc73065e4775905ec44c75f5788b', 'request_id': 'req-9f4456f2-511c-4e40-b9e0-ca65f19c4157'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:38.252734 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-9f4456f2-511c-4e40-b9e0-ca65f19c4157 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa3edc73065e4775905ec44c75f5788b', 'request_id': 'req-9f4456f2-511c-4e40-b9e0-ca65f19c4157'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:38.259094 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-9f4456f2-511c-4e40-b9e0-ca65f19c4157 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa3edc73065e4775905ec44c75f5788b', 'request_id': 'req-9f4456f2-511c-4e40-b9e0-ca65f19c4157'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:38.269897 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-9f4456f2-511c-4e40-b9e0-ca65f19c4157 admin admin] The token cache is being invalidate because project aa3edc73065e4775905ec44c75f5788b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:29:38.270178 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-9f4456f2-511c-4e40-b9e0-ca65f19c4157 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:38.270802 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-9f4456f2-511c-4e40-b9e0-ca65f19c4157 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:38.272997 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1428/2856] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:29:38 2022] DELETE /identity/v3/projects/aa3edc73065e4775905ec44c75f5788b => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 22:29:38.276944 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf239252-98a0-4a10-b9cf-90f9878c4ad2 service neutron] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:38.286419 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf239252-98a0-4a10-b9cf-90f9878c4ad2 service neutron] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:38.342230 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0011cf-c993-41f1-a5ab-5270c063e0e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:38.342842 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0011cf-c993-41f1-a5ab-5270c063e0e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:38.343122 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0011cf-c993-41f1-a5ab-5270c063e0e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:38.343388 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0011cf-c993-41f1-a5ab-5270c063e0e6 admin admin] PATH_INFO: `/v3/users/559ff62257b84d589750bb9277a34b45` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:38.344759 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0011cf-c993-41f1-a5ab-5270c063e0e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=559ff62257b84d589750bb9277a34b45)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:38.346696 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0011cf-c993-41f1-a5ab-5270c063e0e6 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:38.501188 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-0f0011cf-c993-41f1-a5ab-5270c063e0e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '559ff62257b84d589750bb9277a34b45'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:38.512762 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-0f0011cf-c993-41f1-a5ab-5270c063e0e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '559ff62257b84d589750bb9277a34b45'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:38.519754 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-0f0011cf-c993-41f1-a5ab-5270c063e0e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '559ff62257b84d589750bb9277a34b45'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:38.520483 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-0f0011cf-c993-41f1-a5ab-5270c063e0e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '559ff62257b84d589750bb9277a34b45'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:38.521099 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-0f0011cf-c993-41f1-a5ab-5270c063e0e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '559ff62257b84d589750bb9277a34b45'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:38.586091 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1429/2857] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:29:38 2022] DELETE /identity/v3/users/559ff62257b84d589750bb9277a34b45 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:29:38.965382 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4456f2-511c-4e40-b9e0-ca65f19c4157 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:38.969696 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-9f4456f2-511c-4e40-b9e0-ca65f19c4157 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:29:38.983840 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4456f2-511c-4e40-b9e0-ca65f19c4157 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:39.048623 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b88e58-96b4-48af-9498-6d4b7d9790e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:39.049184 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b88e58-96b4-48af-9498-6d4b7d9790e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:39.049461 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b88e58-96b4-48af-9498-6d4b7d9790e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:39.049732 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b88e58-96b4-48af-9498-6d4b7d9790e1 admin admin] PATH_INFO: `/v3/projects/8151ae7deed644e0b83338678586f64a` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:39.050854 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b88e58-96b4-48af-9498-6d4b7d9790e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8151ae7deed644e0b83338678586f64a)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:39.052789 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b88e58-96b4-48af-9498-6d4b7d9790e1 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:39.183388 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-d2b88e58-96b4-48af-9498-6d4b7d9790e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8151ae7deed644e0b83338678586f64a', 'request_id': 'req-d2b88e58-96b4-48af-9498-6d4b7d9790e1'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:39.275581 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-d2b88e58-96b4-48af-9498-6d4b7d9790e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8151ae7deed644e0b83338678586f64a', 'request_id': 'req-d2b88e58-96b4-48af-9498-6d4b7d9790e1'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:39.280243 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:29:39.282231 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-d2b88e58-96b4-48af-9498-6d4b7d9790e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8151ae7deed644e0b83338678586f64a', 'request_id': 'req-d2b88e58-96b4-48af-9498-6d4b7d9790e1'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:39.294463 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-d2b88e58-96b4-48af-9498-6d4b7d9790e1 admin admin] The token cache is being invalidate because project 8151ae7deed644e0b83338678586f64a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:29:39.294751 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-d2b88e58-96b4-48af-9498-6d4b7d9790e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:39.295392 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-d2b88e58-96b4-48af-9498-6d4b7d9790e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:39.296965 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1429/2858] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:29:38 2022] DELETE /identity/v3/projects/8151ae7deed644e0b83338678586f64a => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:29:39.300953 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0011cf-c993-41f1-a5ab-5270c063e0e6 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:39.302359 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-0f0011cf-c993-41f1-a5ab-5270c063e0e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:29:39.315738 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0011cf-c993-41f1-a5ab-5270c063e0e6 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:39.351532 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:29:39.374004 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e22cef5-bc8e-4959-91a6-015ad446cd59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:39.374630 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e22cef5-bc8e-4959-91a6-015ad446cd59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:39.374907 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e22cef5-bc8e-4959-91a6-015ad446cd59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:39.375178 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e22cef5-bc8e-4959-91a6-015ad446cd59 admin admin] PATH_INFO: `/v3/users/fcbcf62fb69a49648ae7023b4eef2585` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:39.376280 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e22cef5-bc8e-4959-91a6-015ad446cd59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fcbcf62fb69a49648ae7023b4eef2585)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:39.378212 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e22cef5-bc8e-4959-91a6-015ad446cd59 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:39.580628 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-4e22cef5-bc8e-4959-91a6-015ad446cd59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fcbcf62fb69a49648ae7023b4eef2585'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:39.592013 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-4e22cef5-bc8e-4959-91a6-015ad446cd59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fcbcf62fb69a49648ae7023b4eef2585'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:39.598857 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-4e22cef5-bc8e-4959-91a6-015ad446cd59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fcbcf62fb69a49648ae7023b4eef2585'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:39.605078 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-4e22cef5-bc8e-4959-91a6-015ad446cd59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fcbcf62fb69a49648ae7023b4eef2585'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:39.605078 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-4e22cef5-bc8e-4959-91a6-015ad446cd59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fcbcf62fb69a49648ae7023b4eef2585'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:39.686535 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1430/2859] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:29:39 2022] DELETE /identity/v3/users/fcbcf62fb69a49648ae7023b4eef2585 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:29:40.079242 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b88e58-96b4-48af-9498-6d4b7d9790e1 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:40.081017 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-d2b88e58-96b4-48af-9498-6d4b7d9790e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:29:40.094625 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b88e58-96b4-48af-9498-6d4b7d9790e1 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:40.157430 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89777af7-c103-4b4a-90b8-33954e09c4c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:40.157988 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89777af7-c103-4b4a-90b8-33954e09c4c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:40.158489 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89777af7-c103-4b4a-90b8-33954e09c4c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:40.158768 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89777af7-c103-4b4a-90b8-33954e09c4c4 admin admin] PATH_INFO: `/v3/projects/a4cf4f56340d4c029d8b33bd5eff6466` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:40.159894 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89777af7-c103-4b4a-90b8-33954e09c4c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4cf4f56340d4c029d8b33bd5eff6466)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:40.161854 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89777af7-c103-4b4a-90b8-33954e09c4c4 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:40.275176 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-89777af7-c103-4b4a-90b8-33954e09c4c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4cf4f56340d4c029d8b33bd5eff6466', 'request_id': 'req-89777af7-c103-4b4a-90b8-33954e09c4c4'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:40.363231 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-89777af7-c103-4b4a-90b8-33954e09c4c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4cf4f56340d4c029d8b33bd5eff6466', 'request_id': 'req-89777af7-c103-4b4a-90b8-33954e09c4c4'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:40.369688 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-89777af7-c103-4b4a-90b8-33954e09c4c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4cf4f56340d4c029d8b33bd5eff6466', 'request_id': 'req-89777af7-c103-4b4a-90b8-33954e09c4c4'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:40.380657 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-89777af7-c103-4b4a-90b8-33954e09c4c4 admin admin] The token cache is being invalidate because project a4cf4f56340d4c029d8b33bd5eff6466 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:29:40.382284 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-89777af7-c103-4b4a-90b8-33954e09c4c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:40.382284 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-89777af7-c103-4b4a-90b8-33954e09c4c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:40.383753 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1430/2860] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:29:40 2022] DELETE /identity/v3/projects/a4cf4f56340d4c029d8b33bd5eff6466 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:29:40.389760 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daeaaae-0015-4f14-a2ca-d3d0debf8fc5 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:40.390235 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daeaaae-0015-4f14-a2ca-d3d0debf8fc5 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:40.390349 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daeaaae-0015-4f14-a2ca-d3d0debf8fc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:40.409300 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-3daeaaae-0015-4f14-a2ca-d3d0debf8fc5 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:29:40.410422 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-3daeaaae-0015-4f14-a2ca-d3d0debf8fc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:29:40.467192 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1431/2861] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:29:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 22:29:40.476324 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838af98f-bb7e-4250-9a43-5702ddfb9a26 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:40.476643 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838af98f-bb7e-4250-9a43-5702ddfb9a26 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:40.476920 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838af98f-bb7e-4250-9a43-5702ddfb9a26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:40.496831 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-838af98f-bb7e-4250-9a43-5702ddfb9a26 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:29:40.498021 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-838af98f-bb7e-4250-9a43-5702ddfb9a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:29:40.505862 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1431/2862] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:29:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:29:40.510027 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daeaaae-0015-4f14-a2ca-d3d0debf8fc5 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:40.519595 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daeaaae-0015-4f14-a2ca-d3d0debf8fc5 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:40.524171 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ed0693-b30c-41a7-89a5-32f23a804ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:40.524792 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ed0693-b30c-41a7-89a5-32f23a804ac8 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:40.525070 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ed0693-b30c-41a7-89a5-32f23a804ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:40.525339 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ed0693-b30c-41a7-89a5-32f23a804ac8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:40.525978 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ed0693-b30c-41a7-89a5-32f23a804ac8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 22:29:40.526278 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ed0693-b30c-41a7-89a5-32f23a804ac8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:40.527869 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ed0693-b30c-41a7-89a5-32f23a804ac8 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:40.539863 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1432/2863] 10.222.0.18 () {58 vars in 1153 bytes} [Sat Aug 6 22:29:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 22:29:40.544123 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838af98f-bb7e-4250-9a43-5702ddfb9a26 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:40.553866 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838af98f-bb7e-4250-9a43-5702ddfb9a26 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:40.558685 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf42e19-35d6-4e51-a3aa-c12499cfa019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:40.559280 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf42e19-35d6-4e51-a3aa-c12499cfa019 admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:40.559556 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf42e19-35d6-4e51-a3aa-c12499cfa019 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:40.559853 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf42e19-35d6-4e51-a3aa-c12499cfa019 admin admin] PATH_INFO: `/v3/projects` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:40.560510 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf42e19-35d6-4e51-a3aa-c12499cfa019 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:40.562441 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf42e19-35d6-4e51-a3aa-c12499cfa019 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:40.650003 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1432/2864] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:29:40 2022] POST /identity/v3/projects => generated 409 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 22:29:40.654277 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ed0693-b30c-41a7-89a5-32f23a804ac8 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:40.668487 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ed0693-b30c-41a7-89a5-32f23a804ac8 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:40.706183 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:29:40.720703 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddf054e-70fb-4d8d-a0b4-1733648fe24c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:40.721301 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddf054e-70fb-4d8d-a0b4-1733648fe24c admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:40.721577 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddf054e-70fb-4d8d-a0b4-1733648fe24c admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:40.721843 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddf054e-70fb-4d8d-a0b4-1733648fe24c admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:40.722523 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddf054e-70fb-4d8d-a0b4-1733648fe24c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:40.724449 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddf054e-70fb-4d8d-a0b4-1733648fe24c admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:40.807809 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1433/2865] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:29:40 2022] POST /identity/v3/users => generated 305 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:29:40.811901 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf42e19-35d6-4e51-a3aa-c12499cfa019 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:40.822408 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf42e19-35d6-4e51-a3aa-c12499cfa019 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:40.827186 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f7a6a8-b579-4aec-b17f-a6910704f14c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:40.827780 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f7a6a8-b579-4aec-b17f-a6910704f14c admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:40.828056 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f7a6a8-b579-4aec-b17f-a6910704f14c admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:40.828324 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f7a6a8-b579-4aec-b17f-a6910704f14c admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:40.829059 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f7a6a8-b579-4aec-b17f-a6910704f14c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:40.830671 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f7a6a8-b579-4aec-b17f-a6910704f14c admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:40.841189 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1433/2866] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:29:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:29:40.845368 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddf054e-70fb-4d8d-a0b4-1733648fe24c admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:40.855822 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddf054e-70fb-4d8d-a0b4-1733648fe24c admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:40.860328 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208e542c-61d2-468e-a3f6-b22525ad2816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:40.860975 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208e542c-61d2-468e-a3f6-b22525ad2816 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:40.861255 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208e542c-61d2-468e-a3f6-b22525ad2816 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:40.861525 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208e542c-61d2-468e-a3f6-b22525ad2816 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:40.862157 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208e542c-61d2-468e-a3f6-b22525ad2816 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:40.863738 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208e542c-61d2-468e-a3f6-b22525ad2816 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:40.874205 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1434/2867] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:29:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:29:40.878093 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f7a6a8-b579-4aec-b17f-a6910704f14c admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:40.889009 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f7a6a8-b579-4aec-b17f-a6910704f14c admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:40.893615 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f796747-27eb-4af7-865f-f6358ed0560d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:40.894251 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f796747-27eb-4af7-865f-f6358ed0560d admin admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:40.894526 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f796747-27eb-4af7-865f-f6358ed0560d admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:40.894794 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f796747-27eb-4af7-865f-f6358ed0560d admin admin] PATH_INFO: `/v3/projects/01cb067e5436412e9689f42aa3b869f1/users/e1e67c2317cf477483959ca83cdfb723/roles/56e4a96094e9434cbbafb22579c034f5` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:40.909437 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f796747-27eb-4af7-865f-f6358ed0560d admin admin] RBAC: Authorizing `identity:create_grant(project_id=01cb067e5436412e9689f42aa3b869f1, user_id=e1e67c2317cf477483959ca83cdfb723, role_id=56e4a96094e9434cbbafb22579c034f5)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:40.912048 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f796747-27eb-4af7-865f-f6358ed0560d admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:40.972200 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1434/2868] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:29:40 2022] PUT /identity/v3/projects/01cb067e5436412e9689f42aa3b869f1/users/e1e67c2317cf477483959ca83cdfb723/roles/56e4a96094e9434cbbafb22579c034f5 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 22:29:40.988854 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac766f00-f4d2-4894-a10e-cf8f57a552ac None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:40.989138 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac766f00-f4d2-4894-a10e-cf8f57a552ac None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:40.989409 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac766f00-f4d2-4894-a10e-cf8f57a552ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:41.015891 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-ac766f00-f4d2-4894-a10e-cf8f57a552ac None None] MFA Rules not processed for user `e1e67c2317cf477483959ca83cdfb723`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:29:41.017227 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-ac766f00-f4d2-4894-a10e-cf8f57a552ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:29:41.079572 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1435/2869] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:29:40 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:29:41.084876 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f796747-27eb-4af7-865f-f6358ed0560d admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:41.103423 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:29:41.145888 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f796747-27eb-4af7-865f-f6358ed0560d admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:41.151036 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adde52f-5fec-48a2-9ea2-74ee54b5f486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:41.151632 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adde52f-5fec-48a2-9ea2-74ee54b5f486 admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:41.151926 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adde52f-5fec-48a2-9ea2-74ee54b5f486 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:41.152198 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adde52f-5fec-48a2-9ea2-74ee54b5f486 admin admin] PATH_INFO: `/v3/projects` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:41.152859 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adde52f-5fec-48a2-9ea2-74ee54b5f486 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:41.154772 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adde52f-5fec-48a2-9ea2-74ee54b5f486 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:41.238147 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1435/2870] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:29:41 2022] POST /identity/v3/projects => generated 409 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:29:41.241920 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac766f00-f4d2-4894-a10e-cf8f57a552ac None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:41.253058 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac766f00-f4d2-4894-a10e-cf8f57a552ac None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:41.308782 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f2cccc-2369-4072-a751-7514bcdf7d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:41.310555 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f2cccc-2369-4072-a751-7514bcdf7d1d admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:41.310835 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f2cccc-2369-4072-a751-7514bcdf7d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:41.311112 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f2cccc-2369-4072-a751-7514bcdf7d1d admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:41.311733 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f2cccc-2369-4072-a751-7514bcdf7d1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:41.313686 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f2cccc-2369-4072-a751-7514bcdf7d1d admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:41.384515 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1436/2871] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:29:41 2022] POST /identity/v3/users => generated 311 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:29:41.388341 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adde52f-5fec-48a2-9ea2-74ee54b5f486 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:41.399397 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adde52f-5fec-48a2-9ea2-74ee54b5f486 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:41.404236 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fae2741-9d50-4adb-9fd0-f3afaeda8412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:41.404701 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fae2741-9d50-4adb-9fd0-f3afaeda8412 admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:41.404986 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fae2741-9d50-4adb-9fd0-f3afaeda8412 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:41.405267 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fae2741-9d50-4adb-9fd0-f3afaeda8412 admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:41.405911 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fae2741-9d50-4adb-9fd0-f3afaeda8412 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:41.407580 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fae2741-9d50-4adb-9fd0-f3afaeda8412 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:41.418747 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1436/2872] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:29:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:29:41.422785 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f2cccc-2369-4072-a751-7514bcdf7d1d admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:41.432798 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f2cccc-2369-4072-a751-7514bcdf7d1d admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:41.437365 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec1763b-b7fd-4712-8f95-dd116fc20832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:41.437998 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec1763b-b7fd-4712-8f95-dd116fc20832 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:41.438277 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec1763b-b7fd-4712-8f95-dd116fc20832 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:41.438547 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec1763b-b7fd-4712-8f95-dd116fc20832 admin admin] PATH_INFO: `/v3/projects/7b0fbce9c78540c6973f5c9f271b2d47/users/27d974f007cf4cc38f098ec87dafa20c/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:41.452953 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec1763b-b7fd-4712-8f95-dd116fc20832 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b0fbce9c78540c6973f5c9f271b2d47, user_id=27d974f007cf4cc38f098ec87dafa20c, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:41.455627 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec1763b-b7fd-4712-8f95-dd116fc20832 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:41.602027 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1437/2873] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:29:41 2022] PUT /identity/v3/projects/7b0fbce9c78540c6973f5c9f271b2d47/users/27d974f007cf4cc38f098ec87dafa20c/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:29:41.606027 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fae2741-9d50-4adb-9fd0-f3afaeda8412 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:41.616081 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fae2741-9d50-4adb-9fd0-f3afaeda8412 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:41.672679 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0258ddf-b483-4a34-9dad-f6156891d18f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:41.673276 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0258ddf-b483-4a34-9dad-f6156891d18f admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:41.673553 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0258ddf-b483-4a34-9dad-f6156891d18f admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:41.673827 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0258ddf-b483-4a34-9dad-f6156891d18f admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:41.674486 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0258ddf-b483-4a34-9dad-f6156891d18f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:41.676079 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0258ddf-b483-4a34-9dad-f6156891d18f admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:41.685905 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1437/2874] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:29:41 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:29:41.690010 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec1763b-b7fd-4712-8f95-dd116fc20832 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:41.700804 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec1763b-b7fd-4712-8f95-dd116fc20832 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:41.705304 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea46afd-3e79-45f9-9c78-07c9adc2a547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:41.705935 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea46afd-3e79-45f9-9c78-07c9adc2a547 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:41.706209 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea46afd-3e79-45f9-9c78-07c9adc2a547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:41.706478 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea46afd-3e79-45f9-9c78-07c9adc2a547 admin admin] PATH_INFO: `/v3/projects/7b0fbce9c78540c6973f5c9f271b2d47/users/27d974f007cf4cc38f098ec87dafa20c/roles/4b8dcf0491fd4072acc69fd894bf77d0` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:41.708358 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea46afd-3e79-45f9-9c78-07c9adc2a547 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b0fbce9c78540c6973f5c9f271b2d47, user_id=27d974f007cf4cc38f098ec87dafa20c, role_id=4b8dcf0491fd4072acc69fd894bf77d0)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:41.711954 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea46afd-3e79-45f9-9c78-07c9adc2a547 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:41.722717 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1438/2875] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:29:41 2022] PUT /identity/v3/projects/7b0fbce9c78540c6973f5c9f271b2d47/users/27d974f007cf4cc38f098ec87dafa20c/roles/4b8dcf0491fd4072acc69fd894bf77d0 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:29:41.734707 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4aa3e9-3818-48f7-9bd6-81ac5ef814c3 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:41.734989 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4aa3e9-3818-48f7-9bd6-81ac5ef814c3 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:41.735290 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4aa3e9-3818-48f7-9bd6-81ac5ef814c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:41.763846 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-6e4aa3e9-3818-48f7-9bd6-81ac5ef814c3 None None] MFA Rules not processed for user `27d974f007cf4cc38f098ec87dafa20c`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:29:41.765085 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-6e4aa3e9-3818-48f7-9bd6-81ac5ef814c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:29:41.832765 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1438/2876] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:29:41 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:29:41.843652 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea46afd-3e79-45f9-9c78-07c9adc2a547 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:41.845625 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-4ea46afd-3e79-45f9-9c78-07c9adc2a547 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:29:41.852698 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:29:41.904220 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea46afd-3e79-45f9-9c78-07c9adc2a547 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:41.908064 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0118216f-5c5a-4fb6-9c1c-cdd5bcac5721 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa71c71de263478da234f3c5716a6e5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:41.908659 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0118216f-5c5a-4fb6-9c1c-cdd5bcac5721 service cinder] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:41.908937 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0118216f-5c5a-4fb6-9c1c-cdd5bcac5721 service cinder] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:41.909203 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0118216f-5c5a-4fb6-9c1c-cdd5bcac5721 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:41.919068 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0118216f-5c5a-4fb6-9c1c-cdd5bcac5721 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:41.920511 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0118216f-5c5a-4fb6-9c1c-cdd5bcac5721 service cinder] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:41.930821 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1439/2877] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:29:41 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:29:42.201307 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4aa3e9-3818-48f7-9bd6-81ac5ef814c3 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:42.211492 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4aa3e9-3818-48f7-9bd6-81ac5ef814c3 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:42.215202 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2536d88-1903-45eb-a516-2b2854212071 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa71c71de263478da234f3c5716a6e5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:42.215789 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2536d88-1903-45eb-a516-2b2854212071 service cinder] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:42.216065 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2536d88-1903-45eb-a516-2b2854212071 service cinder] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:42.216336 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2536d88-1903-45eb-a516-2b2854212071 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:42.224690 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:29:42.274787 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2536d88-1903-45eb-a516-2b2854212071 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:42.276201 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2536d88-1903-45eb-a516-2b2854212071 service cinder] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:42.285205 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1439/2878] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:29:42 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:29:49.821011 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0118216f-5c5a-4fb6-9c1c-cdd5bcac5721 service cinder] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:49.822414 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-0118216f-5c5a-4fb6-9c1c-cdd5bcac5721 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:29:49.891459 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0118216f-5c5a-4fb6-9c1c-cdd5bcac5721 service cinder] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:49.896365 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ce922b-0377-4b84-8cc2-db6b5a9862ca service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '680d62a67ae2456f8c129e26728c3297', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:49.896993 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ce922b-0377-4b84-8cc2-db6b5a9862ca service placement] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:49.897268 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ce922b-0377-4b84-8cc2-db6b5a9862ca service placement] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:49.897534 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ce922b-0377-4b84-8cc2-db6b5a9862ca service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:49.900183 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-52ce922b-0377-4b84-8cc2-db6b5a9862ca service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:29:49.911549 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ce922b-0377-4b84-8cc2-db6b5a9862ca service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:49.913148 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ce922b-0377-4b84-8cc2-db6b5a9862ca service placement] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:49.924847 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1440/2879] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:29:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:29:52.284263 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=23 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:29:54.165480 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2536d88-1903-45eb-a516-2b2854212071 service cinder] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:54.231295 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2536d88-1903-45eb-a516-2b2854212071 service cinder] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:54.235789 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c4c6d6-0be0-4892-87f1-e3e18d3677a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:54.236522 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c4c6d6-0be0-4892-87f1-e3e18d3677a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:54.236835 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c4c6d6-0be0-4892-87f1-e3e18d3677a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:54.237092 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c4c6d6-0be0-4892-87f1-e3e18d3677a3 admin admin] PATH_INFO: `/v3/users/e1e67c2317cf477483959ca83cdfb723` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:54.238206 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c4c6d6-0be0-4892-87f1-e3e18d3677a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1e67c2317cf477483959ca83cdfb723)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:54.240101 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c4c6d6-0be0-4892-87f1-e3e18d3677a3 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:54.393826 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-37c4c6d6-0be0-4892-87f1-e3e18d3677a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1e67c2317cf477483959ca83cdfb723'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:54.406837 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-37c4c6d6-0be0-4892-87f1-e3e18d3677a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1e67c2317cf477483959ca83cdfb723'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:54.454092 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-37c4c6d6-0be0-4892-87f1-e3e18d3677a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1e67c2317cf477483959ca83cdfb723'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:54.461965 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-37c4c6d6-0be0-4892-87f1-e3e18d3677a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1e67c2317cf477483959ca83cdfb723'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:54.465891 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-37c4c6d6-0be0-4892-87f1-e3e18d3677a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1e67c2317cf477483959ca83cdfb723'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:54.468948 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1440/2880] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:29:54 2022] DELETE /identity/v3/users/e1e67c2317cf477483959ca83cdfb723 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 22:29:54.473854 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f016a2-5aa5-4dea-9f83-2e3feff8d7cf None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:54.474343 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f016a2-5aa5-4dea-9f83-2e3feff8d7cf None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:54.474448 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f016a2-5aa5-4dea-9f83-2e3feff8d7cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:54.494335 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-87f016a2-5aa5-4dea-9f83-2e3feff8d7cf None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:29:54.495552 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-87f016a2-5aa5-4dea-9f83-2e3feff8d7cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:29:54.554647 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1441/2881] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:29:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:29:54.564485 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c4c6d6-0be0-4892-87f1-e3e18d3677a3 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:54.565982 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-37c4c6d6-0be0-4892-87f1-e3e18d3677a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:29:54.624395 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c4c6d6-0be0-4892-87f1-e3e18d3677a3 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:54.628114 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dd6742-59bc-449a-9010-51a054ad48a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:54.628751 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dd6742-59bc-449a-9010-51a054ad48a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:54.629030 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dd6742-59bc-449a-9010-51a054ad48a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:54.629299 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dd6742-59bc-449a-9010-51a054ad48a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:54.638222 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dd6742-59bc-449a-9010-51a054ad48a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:54.639634 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dd6742-59bc-449a-9010-51a054ad48a9 service neutron] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:54.648991 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1441/2882] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:29:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:29:55.164768 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f016a2-5aa5-4dea-9f83-2e3feff8d7cf None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:55.166352 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-87f016a2-5aa5-4dea-9f83-2e3feff8d7cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:29:55.180534 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f016a2-5aa5-4dea-9f83-2e3feff8d7cf None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:55.186554 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e341360f-c62e-469a-aaf8-6339adad8839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:55.187288 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e341360f-c62e-469a-aaf8-6339adad8839 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:55.187605 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e341360f-c62e-469a-aaf8-6339adad8839 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:55.187941 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e341360f-c62e-469a-aaf8-6339adad8839 admin admin] PATH_INFO: `/v3/projects/01cb067e5436412e9689f42aa3b869f1` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:55.189845 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e341360f-c62e-469a-aaf8-6339adad8839 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01cb067e5436412e9689f42aa3b869f1)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:55.192180 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e341360f-c62e-469a-aaf8-6339adad8839 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:55.313224 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:29:55.326674 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-e341360f-c62e-469a-aaf8-6339adad8839 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01cb067e5436412e9689f42aa3b869f1', 'request_id': 'req-e341360f-c62e-469a-aaf8-6339adad8839'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:55.410472 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-e341360f-c62e-469a-aaf8-6339adad8839 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01cb067e5436412e9689f42aa3b869f1', 'request_id': 'req-e341360f-c62e-469a-aaf8-6339adad8839'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:55.418787 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-e341360f-c62e-469a-aaf8-6339adad8839 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01cb067e5436412e9689f42aa3b869f1', 'request_id': 'req-e341360f-c62e-469a-aaf8-6339adad8839'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:55.434482 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-e341360f-c62e-469a-aaf8-6339adad8839 admin admin] The token cache is being invalidate because project 01cb067e5436412e9689f42aa3b869f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:29:55.434798 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-e341360f-c62e-469a-aaf8-6339adad8839 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:55.435578 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-e341360f-c62e-469a-aaf8-6339adad8839 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:55.440071 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1442/2883] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:29:55 2022] DELETE /identity/v3/projects/01cb067e5436412e9689f42aa3b869f1 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:29:55.441817 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dd6742-59bc-449a-9010-51a054ad48a9 service neutron] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:55.452078 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dd6742-59bc-449a-9010-51a054ad48a9 service neutron] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:55.473120 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:29:55.528394 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287eaefe-7046-4710-b11a-30d6e0df57b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:55.528900 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287eaefe-7046-4710-b11a-30d6e0df57b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:55.529178 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287eaefe-7046-4710-b11a-30d6e0df57b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:55.529940 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287eaefe-7046-4710-b11a-30d6e0df57b4 admin admin] PATH_INFO: `/v3/users/27d974f007cf4cc38f098ec87dafa20c` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:55.531630 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287eaefe-7046-4710-b11a-30d6e0df57b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27d974f007cf4cc38f098ec87dafa20c)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:55.533683 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287eaefe-7046-4710-b11a-30d6e0df57b4 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:55.767653 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-287eaefe-7046-4710-b11a-30d6e0df57b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27d974f007cf4cc38f098ec87dafa20c'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:55.780361 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-287eaefe-7046-4710-b11a-30d6e0df57b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27d974f007cf4cc38f098ec87dafa20c'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:55.928997 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-287eaefe-7046-4710-b11a-30d6e0df57b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27d974f007cf4cc38f098ec87dafa20c'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:55.936570 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-287eaefe-7046-4710-b11a-30d6e0df57b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27d974f007cf4cc38f098ec87dafa20c'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:55.937251 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-287eaefe-7046-4710-b11a-30d6e0df57b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27d974f007cf4cc38f098ec87dafa20c'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:55.942993 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1442/2884] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:29:55 2022] DELETE /identity/v3/users/27d974f007cf4cc38f098ec87dafa20c => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:29:56.347474 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e341360f-c62e-469a-aaf8-6339adad8839 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:56.352563 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-e341360f-c62e-469a-aaf8-6339adad8839 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:29:56.367418 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e341360f-c62e-469a-aaf8-6339adad8839 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:56.426468 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daff5e6f-d905-4304-9a60-e83ea0369f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:56.427109 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daff5e6f-d905-4304-9a60-e83ea0369f39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:56.427390 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daff5e6f-d905-4304-9a60-e83ea0369f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:56.427663 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daff5e6f-d905-4304-9a60-e83ea0369f39 admin admin] PATH_INFO: `/v3/projects/7b0fbce9c78540c6973f5c9f271b2d47` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:56.428828 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daff5e6f-d905-4304-9a60-e83ea0369f39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b0fbce9c78540c6973f5c9f271b2d47)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:56.430945 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daff5e6f-d905-4304-9a60-e83ea0369f39 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:56.528366 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-daff5e6f-d905-4304-9a60-e83ea0369f39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b0fbce9c78540c6973f5c9f271b2d47', 'request_id': 'req-daff5e6f-d905-4304-9a60-e83ea0369f39'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:56.585173 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-daff5e6f-d905-4304-9a60-e83ea0369f39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b0fbce9c78540c6973f5c9f271b2d47', 'request_id': 'req-daff5e6f-d905-4304-9a60-e83ea0369f39'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:56.591650 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-daff5e6f-d905-4304-9a60-e83ea0369f39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b0fbce9c78540c6973f5c9f271b2d47', 'request_id': 'req-daff5e6f-d905-4304-9a60-e83ea0369f39'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:56.603161 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-daff5e6f-d905-4304-9a60-e83ea0369f39 admin admin] The token cache is being invalidate because project 7b0fbce9c78540c6973f5c9f271b2d47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:29:56.603446 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-daff5e6f-d905-4304-9a60-e83ea0369f39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:56.604112 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-daff5e6f-d905-4304-9a60-e83ea0369f39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:29:56.605611 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1443/2885] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:29:56 2022] DELETE /identity/v3/projects/7b0fbce9c78540c6973f5c9f271b2d47 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:29:56.610486 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7412059-e539-409e-90aa-9ff9e27e030b None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:56.611052 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7412059-e539-409e-90aa-9ff9e27e030b None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:56.611052 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7412059-e539-409e-90aa-9ff9e27e030b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:56.630783 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-b7412059-e539-409e-90aa-9ff9e27e030b None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:29:56.631973 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-b7412059-e539-409e-90aa-9ff9e27e030b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:29:56.689762 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1443/2886] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:29:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:29:56.698668 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8a5244-7874-4a9a-85f8-2d5e8c994598 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:56.698903 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8a5244-7874-4a9a-85f8-2d5e8c994598 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:56.699220 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8a5244-7874-4a9a-85f8-2d5e8c994598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:56.720117 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-3a8a5244-7874-4a9a-85f8-2d5e8c994598 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:29:56.721357 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-3a8a5244-7874-4a9a-85f8-2d5e8c994598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:29:56.729669 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1444/2887] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:29:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:29:56.733714 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7412059-e539-409e-90aa-9ff9e27e030b None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:56.743606 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7412059-e539-409e-90aa-9ff9e27e030b None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:56.748319 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3409ad3a-85f4-4194-933e-76c4b5d17887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:56.748963 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3409ad3a-85f4-4194-933e-76c4b5d17887 admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:56.749246 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3409ad3a-85f4-4194-933e-76c4b5d17887 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:56.749520 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3409ad3a-85f4-4194-933e-76c4b5d17887 admin admin] PATH_INFO: `/v3/domains` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:56.750180 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3409ad3a-85f4-4194-933e-76c4b5d17887 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 22:29:56.750474 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3409ad3a-85f4-4194-933e-76c4b5d17887 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:56.752071 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3409ad3a-85f4-4194-933e-76c4b5d17887 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:56.764553 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1444/2888] 10.222.0.18 () {58 vars in 1153 bytes} [Sat Aug 6 22:29:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 22:29:56.768911 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8a5244-7874-4a9a-85f8-2d5e8c994598 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:56.780290 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8a5244-7874-4a9a-85f8-2d5e8c994598 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:56.785459 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f54fa34-7ca3-4f2b-b7b8-2ff0fb1a6190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:56.785970 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f54fa34-7ca3-4f2b-b7b8-2ff0fb1a6190 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:56.786250 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f54fa34-7ca3-4f2b-b7b8-2ff0fb1a6190 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:56.786524 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f54fa34-7ca3-4f2b-b7b8-2ff0fb1a6190 admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:56.787156 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f54fa34-7ca3-4f2b-b7b8-2ff0fb1a6190 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:56.789108 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f54fa34-7ca3-4f2b-b7b8-2ff0fb1a6190 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:56.890731 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1445/2889] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:29:56 2022] POST /identity/v3/projects => generated 365 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:29:56.894840 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3409ad3a-85f4-4194-933e-76c4b5d17887 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:56.910185 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3409ad3a-85f4-4194-933e-76c4b5d17887 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:56.948681 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:29:56.972479 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fe2eed-f27d-4f06-b3a4-309627acd6ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:56.973089 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fe2eed-f27d-4f06-b3a4-309627acd6ec admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:56.973370 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fe2eed-f27d-4f06-b3a4-309627acd6ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:56.973643 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fe2eed-f27d-4f06-b3a4-309627acd6ec admin admin] PATH_INFO: `/v3/users` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:56.974277 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fe2eed-f27d-4f06-b3a4-309627acd6ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:56.977980 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fe2eed-f27d-4f06-b3a4-309627acd6ec admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:57.092678 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1445/2890] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:29:56 2022] POST /identity/v3/users => generated 283 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:29:57.099290 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f54fa34-7ca3-4f2b-b7b8-2ff0fb1a6190 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:57.110320 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f54fa34-7ca3-4f2b-b7b8-2ff0fb1a6190 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:57.114821 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5601aa-0806-4ce9-ac3f-9f0875eda55d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:57.115417 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5601aa-0806-4ce9-ac3f-9f0875eda55d admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:57.115693 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5601aa-0806-4ce9-ac3f-9f0875eda55d admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:57.115963 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5601aa-0806-4ce9-ac3f-9f0875eda55d admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:57.116637 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5601aa-0806-4ce9-ac3f-9f0875eda55d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:57.118269 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5601aa-0806-4ce9-ac3f-9f0875eda55d admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:57.128022 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1446/2891] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:29:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:29:57.132006 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fe2eed-f27d-4f06-b3a4-309627acd6ec admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:57.142160 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fe2eed-f27d-4f06-b3a4-309627acd6ec admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:57.146756 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7d4096-2e11-48bb-a4f4-6697d919143e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:57.147346 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7d4096-2e11-48bb-a4f4-6697d919143e admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:57.147620 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7d4096-2e11-48bb-a4f4-6697d919143e admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:57.148472 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7d4096-2e11-48bb-a4f4-6697d919143e admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:57.149162 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7d4096-2e11-48bb-a4f4-6697d919143e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:57.150754 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7d4096-2e11-48bb-a4f4-6697d919143e admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:57.160863 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1446/2892] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:29:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:29:57.165453 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5601aa-0806-4ce9-ac3f-9f0875eda55d admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:57.175331 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5601aa-0806-4ce9-ac3f-9f0875eda55d admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:57.180014 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff225a1d-50ce-46cb-b096-0d16a5d1f50b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:57.180705 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff225a1d-50ce-46cb-b096-0d16a5d1f50b admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:57.180983 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff225a1d-50ce-46cb-b096-0d16a5d1f50b admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:57.181252 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff225a1d-50ce-46cb-b096-0d16a5d1f50b admin admin] PATH_INFO: `/v3/projects/677ea97512b34f8bbc4a28d4e1e25eb2/users/922a1a794e634f3382e1fbe1be79910e/roles/56e4a96094e9434cbbafb22579c034f5` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:57.189732 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:29:57.196537 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff225a1d-50ce-46cb-b096-0d16a5d1f50b admin admin] RBAC: Authorizing `identity:create_grant(project_id=677ea97512b34f8bbc4a28d4e1e25eb2, user_id=922a1a794e634f3382e1fbe1be79910e, role_id=56e4a96094e9434cbbafb22579c034f5)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:57.199285 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff225a1d-50ce-46cb-b096-0d16a5d1f50b admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:57.288235 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1447/2893] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:29:57 2022] PUT /identity/v3/projects/677ea97512b34f8bbc4a28d4e1e25eb2/users/922a1a794e634f3382e1fbe1be79910e/roles/56e4a96094e9434cbbafb22579c034f5 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:29:57.300459 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccdb931-f253-4d22-8770-5206e76caacb None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:57.300742 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccdb931-f253-4d22-8770-5206e76caacb None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:57.301013 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccdb931-f253-4d22-8770-5206e76caacb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:57.330158 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-cccdb931-f253-4d22-8770-5206e76caacb None None] MFA Rules not processed for user `922a1a794e634f3382e1fbe1be79910e`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:29:57.331406 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-cccdb931-f253-4d22-8770-5206e76caacb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:29:57.397194 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1447/2894] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:29:57 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:29:57.409510 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff225a1d-50ce-46cb-b096-0d16a5d1f50b admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:29:57.411242 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-ff225a1d-50ce-46cb-b096-0d16a5d1f50b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:29:57.469921 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff225a1d-50ce-46cb-b096-0d16a5d1f50b admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:29:57.473516 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f02d881-ac3f-481d-a1ed-49c49daee48f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa71c71de263478da234f3c5716a6e5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:29:57.474117 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f02d881-ac3f-481d-a1ed-49c49daee48f service cinder] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:29:57.474397 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f02d881-ac3f-481d-a1ed-49c49daee48f service cinder] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:29:57.474662 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f02d881-ac3f-481d-a1ed-49c49daee48f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:29:57.483182 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f02d881-ac3f-481d-a1ed-49c49daee48f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:29:57.484707 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f02d881-ac3f-481d-a1ed-49c49daee48f service cinder] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:29:57.494240 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1448/2895] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:29:57 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:30:02.390519 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccdb931-f253-4d22-8770-5206e76caacb None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:30:02.391970 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-cccdb931-f253-4d22-8770-5206e76caacb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:30:02.461423 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccdb931-f253-4d22-8770-5206e76caacb None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:30:02.465231 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4f2358-7e5b-429b-a7e0-9537805d9b80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:30:02.465847 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4f2358-7e5b-429b-a7e0-9537805d9b80 service neutron] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:30:02.466128 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4f2358-7e5b-429b-a7e0-9537805d9b80 service neutron] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:30:02.466402 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4f2358-7e5b-429b-a7e0-9537805d9b80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:30:02.468177 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-ce4f2358-7e5b-429b-a7e0-9537805d9b80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:30:02.514081 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:30:02.537602 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4f2358-7e5b-429b-a7e0-9537805d9b80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:30:02.539727 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4f2358-7e5b-429b-a7e0-9537805d9b80 service neutron] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:30:02.550739 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1448/2896] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:30:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:30:07.489622 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=31,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:30:10.098861 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f02d881-ac3f-481d-a1ed-49c49daee48f service cinder] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:30:10.100487 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-2f02d881-ac3f-481d-a1ed-49c49daee48f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:30:10.115608 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f02d881-ac3f-481d-a1ed-49c49daee48f service cinder] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:30:10.121087 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009a6fef-c2cd-4944-a0e6-b4b6a6aebaf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:30:10.125372 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009a6fef-c2cd-4944-a0e6-b4b6a6aebaf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:30:10.125372 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009a6fef-c2cd-4944-a0e6-b4b6a6aebaf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:30:10.125372 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009a6fef-c2cd-4944-a0e6-b4b6a6aebaf2 admin admin] PATH_INFO: `/v3/users/2dae6a7ec4404626b75a6702386d091b` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:30:10.125372 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009a6fef-c2cd-4944-a0e6-b4b6a6aebaf2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2dae6a7ec4404626b75a6702386d091b)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:30:10.125372 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009a6fef-c2cd-4944-a0e6-b4b6a6aebaf2 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:30:10.277927 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-009a6fef-c2cd-4944-a0e6-b4b6a6aebaf2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2dae6a7ec4404626b75a6702386d091b'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:30:10.290278 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-009a6fef-c2cd-4944-a0e6-b4b6a6aebaf2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2dae6a7ec4404626b75a6702386d091b'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:30:10.297065 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-009a6fef-c2cd-4944-a0e6-b4b6a6aebaf2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2dae6a7ec4404626b75a6702386d091b'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:30:10.300110 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-009a6fef-c2cd-4944-a0e6-b4b6a6aebaf2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2dae6a7ec4404626b75a6702386d091b'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:30:10.300110 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-009a6fef-c2cd-4944-a0e6-b4b6a6aebaf2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2dae6a7ec4404626b75a6702386d091b'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:30:10.379522 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1449/2897] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:30:10 2022] DELETE /identity/v3/users/2dae6a7ec4404626b75a6702386d091b => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:30:10.647783 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4f2358-7e5b-429b-a7e0-9537805d9b80 service neutron] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:30:10.667273 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4f2358-7e5b-429b-a7e0-9537805d9b80 service neutron] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:30:10.722818 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4881ce65-619a-4a72-b147-564ea1715675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:30:10.723445 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4881ce65-619a-4a72-b147-564ea1715675 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:30:10.723722 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4881ce65-619a-4a72-b147-564ea1715675 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:30:10.723992 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4881ce65-619a-4a72-b147-564ea1715675 admin admin] PATH_INFO: `/v3/projects/8168ba3e30a743a582d1c182ca2423b3` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:30:10.725136 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4881ce65-619a-4a72-b147-564ea1715675 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8168ba3e30a743a582d1c182ca2423b3)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:30:10.727671 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4881ce65-619a-4a72-b147-564ea1715675 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:30:10.859700 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-4881ce65-619a-4a72-b147-564ea1715675 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8168ba3e30a743a582d1c182ca2423b3', 'request_id': 'req-4881ce65-619a-4a72-b147-564ea1715675'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:30:10.933479 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-4881ce65-619a-4a72-b147-564ea1715675 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8168ba3e30a743a582d1c182ca2423b3', 'request_id': 'req-4881ce65-619a-4a72-b147-564ea1715675'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:30:10.939415 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-4881ce65-619a-4a72-b147-564ea1715675 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8168ba3e30a743a582d1c182ca2423b3', 'request_id': 'req-4881ce65-619a-4a72-b147-564ea1715675'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:30:10.951518 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-4881ce65-619a-4a72-b147-564ea1715675 admin admin] The token cache is being invalidate because project 8168ba3e30a743a582d1c182ca2423b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:30:10.951805 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-4881ce65-619a-4a72-b147-564ea1715675 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:30:10.952458 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-4881ce65-619a-4a72-b147-564ea1715675 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:30:10.953958 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1449/2898] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:30:10 2022] DELETE /identity/v3/projects/8168ba3e30a743a582d1c182ca2423b3 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:30:10.957983 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009a6fef-c2cd-4944-a0e6-b4b6a6aebaf2 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:30:10.959345 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-009a6fef-c2cd-4944-a0e6-b4b6a6aebaf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:30:10.973006 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009a6fef-c2cd-4944-a0e6-b4b6a6aebaf2 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:30:11.028232 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b39ca86-5456-42f0-8e16-885148f4da6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:30:11.028879 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b39ca86-5456-42f0-8e16-885148f4da6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:30:11.029150 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b39ca86-5456-42f0-8e16-885148f4da6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:30:11.029416 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b39ca86-5456-42f0-8e16-885148f4da6b admin admin] PATH_INFO: `/v3/users/3666a532a7a04fde9c3ff59d894556d4` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:30:11.030535 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b39ca86-5456-42f0-8e16-885148f4da6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3666a532a7a04fde9c3ff59d894556d4)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:30:11.032436 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b39ca86-5456-42f0-8e16-885148f4da6b admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:30:11.217243 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-4b39ca86-5456-42f0-8e16-885148f4da6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3666a532a7a04fde9c3ff59d894556d4'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:30:11.229899 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-4b39ca86-5456-42f0-8e16-885148f4da6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3666a532a7a04fde9c3ff59d894556d4'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:30:11.236798 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-4b39ca86-5456-42f0-8e16-885148f4da6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3666a532a7a04fde9c3ff59d894556d4'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:30:11.237472 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-4b39ca86-5456-42f0-8e16-885148f4da6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3666a532a7a04fde9c3ff59d894556d4'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:30:11.238335 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-4b39ca86-5456-42f0-8e16-885148f4da6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3666a532a7a04fde9c3ff59d894556d4'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:30:11.239715 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:30:11.311245 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1450/2899] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:30:10 2022] DELETE /identity/v3/users/3666a532a7a04fde9c3ff59d894556d4 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:30:11.569904 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4881ce65-619a-4a72-b147-564ea1715675 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:30:11.571019 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-4881ce65-619a-4a72-b147-564ea1715675 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:30:11.586319 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4881ce65-619a-4a72-b147-564ea1715675 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:30:11.653187 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd769c9-ed12-4316-a0b1-539447cb37fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:30:11.653863 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd769c9-ed12-4316-a0b1-539447cb37fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:30:11.654088 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd769c9-ed12-4316-a0b1-539447cb37fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:30:11.654413 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd769c9-ed12-4316-a0b1-539447cb37fb admin admin] PATH_INFO: `/v3/projects/dc93cf73f8474c6b96faa59cfd61c850` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:30:11.655677 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd769c9-ed12-4316-a0b1-539447cb37fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc93cf73f8474c6b96faa59cfd61c850)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:30:11.657690 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd769c9-ed12-4316-a0b1-539447cb37fb admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:30:11.666598 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:30:11.775149 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-ecd769c9-ed12-4316-a0b1-539447cb37fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc93cf73f8474c6b96faa59cfd61c850', 'request_id': 'req-ecd769c9-ed12-4316-a0b1-539447cb37fb'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:30:11.849177 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-ecd769c9-ed12-4316-a0b1-539447cb37fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc93cf73f8474c6b96faa59cfd61c850', 'request_id': 'req-ecd769c9-ed12-4316-a0b1-539447cb37fb'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:30:11.855470 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-ecd769c9-ed12-4316-a0b1-539447cb37fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc93cf73f8474c6b96faa59cfd61c850', 'request_id': 'req-ecd769c9-ed12-4316-a0b1-539447cb37fb'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:30:11.867808 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-ecd769c9-ed12-4316-a0b1-539447cb37fb admin admin] The token cache is being invalidate because project dc93cf73f8474c6b96faa59cfd61c850 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:30:11.868185 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-ecd769c9-ed12-4316-a0b1-539447cb37fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:30:11.868877 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-ecd769c9-ed12-4316-a0b1-539447cb37fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:30:11.881131 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1450/2900] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:30:11 2022] DELETE /identity/v3/projects/dc93cf73f8474c6b96faa59cfd61c850 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:30:21.346450 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=2 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:30:21.864642 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=21,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:30:43.389315 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b39ca86-5456-42f0-8e16-885148f4da6b admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:30:43.390732 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-4b39ca86-5456-42f0-8e16-885148f4da6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:30:43.404193 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b39ca86-5456-42f0-8e16-885148f4da6b admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:30:43.461176 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e522a26-14fb-4a5f-9033-693901662880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:30:43.461796 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e522a26-14fb-4a5f-9033-693901662880 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:30:43.462160 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e522a26-14fb-4a5f-9033-693901662880 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:30:43.462444 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e522a26-14fb-4a5f-9033-693901662880 admin admin] PATH_INFO: `/v3/users/922a1a794e634f3382e1fbe1be79910e` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:30:43.463526 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e522a26-14fb-4a5f-9033-693901662880 admin admin] RBAC: Authorizing `identity:delete_user(user_id=922a1a794e634f3382e1fbe1be79910e)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:30:43.465475 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e522a26-14fb-4a5f-9033-693901662880 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:30:43.665222 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-2e522a26-14fb-4a5f-9033-693901662880 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '922a1a794e634f3382e1fbe1be79910e'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:30:43.677345 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-2e522a26-14fb-4a5f-9033-693901662880 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '922a1a794e634f3382e1fbe1be79910e'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:30:43.684005 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-2e522a26-14fb-4a5f-9033-693901662880 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '922a1a794e634f3382e1fbe1be79910e'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:30:43.684686 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-2e522a26-14fb-4a5f-9033-693901662880 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '922a1a794e634f3382e1fbe1be79910e'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:30:43.685422 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-2e522a26-14fb-4a5f-9033-693901662880 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '922a1a794e634f3382e1fbe1be79910e'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:30:43.747175 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1451/2901] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:30:43 2022] DELETE /identity/v3/users/922a1a794e634f3382e1fbe1be79910e => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:30:43.752497 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c413f-8e69-4dd1-8735-6284d2afbb50 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:30:43.753090 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c413f-8e69-4dd1-8735-6284d2afbb50 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:30:43.753090 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c413f-8e69-4dd1-8735-6284d2afbb50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:30:43.781841 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-fa4c413f-8e69-4dd1-8735-6284d2afbb50 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:30:43.783054 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-fa4c413f-8e69-4dd1-8735-6284d2afbb50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:30:43.841706 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1451/2902] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:30:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:30:43.851906 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e522a26-14fb-4a5f-9033-693901662880 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:30:43.853386 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-2e522a26-14fb-4a5f-9033-693901662880 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:30:43.909311 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e522a26-14fb-4a5f-9033-693901662880 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:30:43.912886 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f392b0-5a15-44bb-80fb-7e69f81bf3c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:30:43.913478 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f392b0-5a15-44bb-80fb-7e69f81bf3c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:30:43.913757 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f392b0-5a15-44bb-80fb-7e69f81bf3c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:30:43.914025 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f392b0-5a15-44bb-80fb-7e69f81bf3c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:30:43.923446 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f392b0-5a15-44bb-80fb-7e69f81bf3c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:30:43.924839 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f392b0-5a15-44bb-80fb-7e69f81bf3c3 service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:30:43.933985 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1452/2903] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:30:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:30:44.322681 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4c413f-8e69-4dd1-8735-6284d2afbb50 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:30:44.324163 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-fa4c413f-8e69-4dd1-8735-6284d2afbb50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:30:44.336574 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4c413f-8e69-4dd1-8735-6284d2afbb50 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:30:44.341437 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f651bd96-f597-429f-adff-2369d0d925f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:30:44.341935 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f651bd96-f597-429f-adff-2369d0d925f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:30:44.342212 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f651bd96-f597-429f-adff-2369d0d925f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:30:44.342566 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f651bd96-f597-429f-adff-2369d0d925f4 admin admin] PATH_INFO: `/v3/projects/677ea97512b34f8bbc4a28d4e1e25eb2` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:30:44.343780 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f651bd96-f597-429f-adff-2369d0d925f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=677ea97512b34f8bbc4a28d4e1e25eb2)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:30:44.346492 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f651bd96-f597-429f-adff-2369d0d925f4 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:30:44.476643 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-f651bd96-f597-429f-adff-2369d0d925f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '677ea97512b34f8bbc4a28d4e1e25eb2', 'request_id': 'req-f651bd96-f597-429f-adff-2369d0d925f4'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:30:44.549295 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-f651bd96-f597-429f-adff-2369d0d925f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '677ea97512b34f8bbc4a28d4e1e25eb2', 'request_id': 'req-f651bd96-f597-429f-adff-2369d0d925f4'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:30:44.555907 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-f651bd96-f597-429f-adff-2369d0d925f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '677ea97512b34f8bbc4a28d4e1e25eb2', 'request_id': 'req-f651bd96-f597-429f-adff-2369d0d925f4'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:30:44.567320 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-f651bd96-f597-429f-adff-2369d0d925f4 admin admin] The token cache is being invalidate because project 677ea97512b34f8bbc4a28d4e1e25eb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:30:44.567612 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-f651bd96-f597-429f-adff-2369d0d925f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:30:44.568242 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-f651bd96-f597-429f-adff-2369d0d925f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:30:44.569757 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1452/2904] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:30:44 2022] DELETE /identity/v3/projects/677ea97512b34f8bbc4a28d4e1e25eb2 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:30:44.574703 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a39b680-7d07-4221-bf82-e0f91bc2e704 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:30:44.575190 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a39b680-7d07-4221-bf82-e0f91bc2e704 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:30:44.575314 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a39b680-7d07-4221-bf82-e0f91bc2e704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:30:44.608865 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:30:44.614273 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-4a39b680-7d07-4221-bf82-e0f91bc2e704 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:30:44.615472 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-4a39b680-7d07-4221-bf82-e0f91bc2e704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:30:44.674000 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1453/2905] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:30:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:30:44.681813 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9361f249-b294-4d98-9549-97fc6e938ecc None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:30:44.687429 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9361f249-b294-4d98-9549-97fc6e938ecc None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:30:44.687429 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9361f249-b294-4d98-9549-97fc6e938ecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:30:44.703032 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-9361f249-b294-4d98-9549-97fc6e938ecc None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:30:44.704940 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-9361f249-b294-4d98-9549-97fc6e938ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:30:44.712748 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1453/2906] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:30:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:30:44.716909 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a39b680-7d07-4221-bf82-e0f91bc2e704 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:30:44.727580 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a39b680-7d07-4221-bf82-e0f91bc2e704 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:30:44.732050 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c8ad09-d31a-44bd-84df-d93925d5806a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:30:44.732673 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c8ad09-d31a-44bd-84df-d93925d5806a admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:30:44.732955 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c8ad09-d31a-44bd-84df-d93925d5806a admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:30:44.733229 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c8ad09-d31a-44bd-84df-d93925d5806a admin admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:30:44.733870 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c8ad09-d31a-44bd-84df-d93925d5806a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 22:30:44.734172 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c8ad09-d31a-44bd-84df-d93925d5806a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:30:44.735797 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c8ad09-d31a-44bd-84df-d93925d5806a admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:30:44.747140 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1454/2907] 10.222.0.18 () {58 vars in 1153 bytes} [Sat Aug 6 22:30:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 22:30:44.751841 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9361f249-b294-4d98-9549-97fc6e938ecc None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:30:44.761021 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9361f249-b294-4d98-9549-97fc6e938ecc None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:30:44.765776 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f54f30d-de44-4f86-a62f-ccfa6feb4bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:30:44.766990 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f54f30d-de44-4f86-a62f-ccfa6feb4bbd admin admin] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:30:44.767563 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f54f30d-de44-4f86-a62f-ccfa6feb4bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:30:44.767563 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f54f30d-de44-4f86-a62f-ccfa6feb4bbd admin admin] PATH_INFO: `/v3/projects` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:30:44.768156 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f54f30d-de44-4f86-a62f-ccfa6feb4bbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:30:44.770087 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f54f30d-de44-4f86-a62f-ccfa6feb4bbd admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:30:44.898469 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1454/2908] 10.222.0.18 () {60 vars in 1153 bytes} [Sat Aug 6 22:30:44 2022] POST /identity/v3/projects => generated 371 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:30:44.902459 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c8ad09-d31a-44bd-84df-d93925d5806a admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:30:44.912369 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c8ad09-d31a-44bd-84df-d93925d5806a admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:30:44.966885 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f48579-7531-4d13-b2ed-52579448aa41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:30:44.967472 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f48579-7531-4d13-b2ed-52579448aa41 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:30:44.967746 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f48579-7531-4d13-b2ed-52579448aa41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:30:44.968013 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f48579-7531-4d13-b2ed-52579448aa41 admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:30:44.968647 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f48579-7531-4d13-b2ed-52579448aa41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:30:44.971120 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f48579-7531-4d13-b2ed-52579448aa41 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:30:45.050418 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1455/2909] 10.222.0.18 () {60 vars in 1144 bytes} [Sat Aug 6 22:30:44 2022] POST /identity/v3/users => generated 286 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 22:30:45.054433 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f54f30d-de44-4f86-a62f-ccfa6feb4bbd admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:30:45.065299 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f54f30d-de44-4f86-a62f-ccfa6feb4bbd admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:30:45.069978 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2b28fe-3747-443e-962d-d69ee7261793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:30:45.070572 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b28fe-3747-443e-962d-d69ee7261793 admin admin] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:30:45.070852 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b28fe-3747-443e-962d-d69ee7261793 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:30:45.071125 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b28fe-3747-443e-962d-d69ee7261793 admin admin] PATH_INFO: `/v3/roles` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:30:45.071800 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2b28fe-3747-443e-962d-d69ee7261793 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:30:45.073456 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2b28fe-3747-443e-962d-d69ee7261793 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:30:45.083948 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1455/2910] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:30:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:30:45.087884 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f48579-7531-4d13-b2ed-52579448aa41 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:30:45.098339 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f48579-7531-4d13-b2ed-52579448aa41 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:30:45.102911 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf9e285-9dfb-4d38-b76f-67d76b6e6652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:30:45.103497 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf9e285-9dfb-4d38-b76f-67d76b6e6652 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:30:45.103775 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf9e285-9dfb-4d38-b76f-67d76b6e6652 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:30:45.104047 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf9e285-9dfb-4d38-b76f-67d76b6e6652 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:30:45.104723 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf9e285-9dfb-4d38-b76f-67d76b6e6652 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:30:45.106311 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf9e285-9dfb-4d38-b76f-67d76b6e6652 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:30:45.117251 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1456/2911] 10.222.0.18 () {58 vars in 1122 bytes} [Sat Aug 6 22:30:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 22:30:45.121253 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2b28fe-3747-443e-962d-d69ee7261793 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:30:45.131781 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2b28fe-3747-443e-962d-d69ee7261793 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:30:45.136388 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1e973f-eeed-46c2-9d6d-af4372ecc144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:30:45.137054 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e973f-eeed-46c2-9d6d-af4372ecc144 admin admin] REQUEST_METHOD: `PUT` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:30:45.137335 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e973f-eeed-46c2-9d6d-af4372ecc144 admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:30:45.137611 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e973f-eeed-46c2-9d6d-af4372ecc144 admin admin] PATH_INFO: `/v3/projects/37f3359917bb4cf099b5daa3607dd6a0/users/e10ddc6237494c98ba81691a3a202f69/roles/56e4a96094e9434cbbafb22579c034f5` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:30:45.151938 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1e973f-eeed-46c2-9d6d-af4372ecc144 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37f3359917bb4cf099b5daa3607dd6a0, user_id=e10ddc6237494c98ba81691a3a202f69, role_id=56e4a96094e9434cbbafb22579c034f5)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:30:45.154628 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1e973f-eeed-46c2-9d6d-af4372ecc144 admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:30:45.232974 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1456/2912] 10.222.0.18 () {60 vars in 1483 bytes} [Sat Aug 6 22:30:45 2022] PUT /identity/v3/projects/37f3359917bb4cf099b5daa3607dd6a0/users/e10ddc6237494c98ba81691a3a202f69/roles/56e4a96094e9434cbbafb22579c034f5 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:30:45.245183 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b9bd3a-0b50-42f1-92e3-b5395c67d49d None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:30:45.245466 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b9bd3a-0b50-42f1-92e3-b5395c67d49d None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:30:45.245741 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b9bd3a-0b50-42f1-92e3-b5395c67d49d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:30:45.273172 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-56b9bd3a-0b50-42f1-92e3-b5395c67d49d None None] MFA Rules not processed for user `e10ddc6237494c98ba81691a3a202f69`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:30:45.274370 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-56b9bd3a-0b50-42f1-92e3-b5395c67d49d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:30:45.336459 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1457/2913] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:30:45 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:30:45.348068 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1e973f-eeed-46c2-9d6d-af4372ecc144 admin admin] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:30:45.349835 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-7b1e973f-eeed-46c2-9d6d-af4372ecc144 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:30:45.354759 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:30:45.430827 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1e973f-eeed-46c2-9d6d-af4372ecc144 admin admin] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:30:45.434551 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8568ca09-7f0b-4259-89ea-227b73cfdae0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa71c71de263478da234f3c5716a6e5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:30:45.438502 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8568ca09-7f0b-4259-89ea-227b73cfdae0 service cinder] REQUEST_METHOD: `GET` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:30:45.438502 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8568ca09-7f0b-4259-89ea-227b73cfdae0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:30:45.438502 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8568ca09-7f0b-4259-89ea-227b73cfdae0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:30:45.444815 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8568ca09-7f0b-4259-89ea-227b73cfdae0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:30:45.446406 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8568ca09-7f0b-4259-89ea-227b73cfdae0 service cinder] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:30:45.455202 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1457/2914] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:30:45 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:30:45.480105 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b9bd3a-0b50-42f1-92e3-b5395c67d49d None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:30:45.484252 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:30:45.543601 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b9bd3a-0b50-42f1-92e3-b5395c67d49d None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:30:45.548891 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681a06c1-cec0-46d4-af54-4fdcd55b612c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:30:45.549508 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681a06c1-cec0-46d4-af54-4fdcd55b612c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:30:45.549786 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681a06c1-cec0-46d4-af54-4fdcd55b612c admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:30:45.550054 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681a06c1-cec0-46d4-af54-4fdcd55b612c admin admin] PATH_INFO: `/v3/users/e10ddc6237494c98ba81691a3a202f69` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:30:45.551169 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681a06c1-cec0-46d4-af54-4fdcd55b612c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e10ddc6237494c98ba81691a3a202f69)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:30:45.553168 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681a06c1-cec0-46d4-af54-4fdcd55b612c admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:30:45.801753 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-681a06c1-cec0-46d4-af54-4fdcd55b612c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e10ddc6237494c98ba81691a3a202f69'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:30:45.813369 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-681a06c1-cec0-46d4-af54-4fdcd55b612c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e10ddc6237494c98ba81691a3a202f69'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:30:45.820118 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-681a06c1-cec0-46d4-af54-4fdcd55b612c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e10ddc6237494c98ba81691a3a202f69'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:30:45.822640 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-681a06c1-cec0-46d4-af54-4fdcd55b612c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e10ddc6237494c98ba81691a3a202f69'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:30:45.823252 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-681a06c1-cec0-46d4-af54-4fdcd55b612c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e10ddc6237494c98ba81691a3a202f69'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:30:45.896794 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1458/2915] 10.222.0.18 () {58 vars in 1224 bytes} [Sat Aug 6 22:30:45 2022] DELETE /identity/v3/users/e10ddc6237494c98ba81691a3a202f69 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:30:45.901442 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1081a1a0-e326-4999-8c5e-92069bcaf2d7 None None] REQUEST_METHOD: `POST` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:30:45.901974 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1081a1a0-e326-4999-8c5e-92069bcaf2d7 None None] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:30:45.901974 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1081a1a0-e326-4999-8c5e-92069bcaf2d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:30:45.921370 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.auth.core [None req-1081a1a0-e326-4999-8c5e-92069bcaf2d7 None None] MFA Rules not processed for user `6bb7ac25afcb4fcaba91aca891ca7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 22:30:45.922895 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-1081a1a0-e326-4999-8c5e-92069bcaf2d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:30:45.981586 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1458/2916] 10.222.0.18 () {58 vars in 958 bytes} [Sat Aug 6 22:30:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 22:30:45.991308 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681a06c1-cec0-46d4-af54-4fdcd55b612c admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:30:45.992740 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-681a06c1-cec0-46d4-af54-4fdcd55b612c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:30:46.049027 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681a06c1-cec0-46d4-af54-4fdcd55b612c admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:30:46.052539 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea05534-a942-4339-9dff-ee0bede27798 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ceaeadc843c4186b25114b28607fe22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:30:46.053142 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea05534-a942-4339-9dff-ee0bede27798 service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:30:46.053421 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea05534-a942-4339-9dff-ee0bede27798 service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:30:46.053696 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea05534-a942-4339-9dff-ee0bede27798 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:30:46.062661 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea05534-a942-4339-9dff-ee0bede27798 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:30:46.064139 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea05534-a942-4339-9dff-ee0bede27798 service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:30:46.073530 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1459/2917] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:30:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:30:46.492270 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1081a1a0-e326-4999-8c5e-92069bcaf2d7 None None] Authenticating user token {{(pid=80423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:30:46.493798 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.fernet_utils [None req-1081a1a0-e326-4999-8c5e-92069bcaf2d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:30:46.507597 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1081a1a0-e326-4999-8c5e-92069bcaf2d7 None None] Validating token access rules against request {{(pid=80423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:30:46.512426 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad21fea-38d1-40cf-9513-602bedf0ae3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb7ac25afcb4fcaba91aca891ca7fc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c285ce4925f04fb5acce87c789a71e83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:30:46.513072 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad21fea-38d1-40cf-9513-602bedf0ae3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:30:46.513376 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad21fea-38d1-40cf-9513-602bedf0ae3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:30:46.513702 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad21fea-38d1-40cf-9513-602bedf0ae3e admin admin] PATH_INFO: `/v3/projects/37f3359917bb4cf099b5daa3607dd6a0` {{(pid=80423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:30:46.514856 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad21fea-38d1-40cf-9513-602bedf0ae3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=37f3359917bb4cf099b5daa3607dd6a0)` {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:30:46.517911 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad21fea-38d1-40cf-9513-602bedf0ae3e admin admin] RBAC: Authorization granted {{(pid=80423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:30:46.629798 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-5ad21fea-38d1-40cf-9513-602bedf0ae3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37f3359917bb4cf099b5daa3607dd6a0', 'request_id': 'req-5ad21fea-38d1-40cf-9513-602bedf0ae3e'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:30:46.686162 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-5ad21fea-38d1-40cf-9513-602bedf0ae3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37f3359917bb4cf099b5daa3607dd6a0', 'request_id': 'req-5ad21fea-38d1-40cf-9513-602bedf0ae3e'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:30:46.692779 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-5ad21fea-38d1-40cf-9513-602bedf0ae3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37f3359917bb4cf099b5daa3607dd6a0', 'request_id': 'req-5ad21fea-38d1-40cf-9513-602bedf0ae3e'} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:30:46.703888 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-5ad21fea-38d1-40cf-9513-602bedf0ae3e admin admin] The token cache is being invalidate because project 37f3359917bb4cf099b5daa3607dd6a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 22:30:46.704173 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-5ad21fea-38d1-40cf-9513-602bedf0ae3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:30:46.704836 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG keystone.notifications [None req-5ad21fea-38d1-40cf-9513-602bedf0ae3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 22:30:46.706346 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: [pid: 80423|app: 0|req: 1459/2918] 10.222.0.18 () {58 vars in 1233 bytes} [Sat Aug 6 22:30:46 2022] DELETE /identity/v3/projects/37f3359917bb4cf099b5daa3607dd6a0 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 22:30:56.069440 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=78,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:30:56.700847 dv-cinder-849846-7-iscsi devstack@keystone.service[80423]: DEBUG dbcounter [-] [80423] Writing DB stats keystone:SELECT=87,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 22:34:53.212855 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea05534-a942-4339-9dff-ee0bede27798 service neutron] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 22:34:53.214255 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-bea05534-a942-4339-9dff-ee0bede27798 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:34:53.282572 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea05534-a942-4339-9dff-ee0bede27798 service neutron] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 22:34:53.340933 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dc7449-fdf0-4ef7-9055-197d5bcad0f6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '680d62a67ae2456f8c129e26728c3297', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb6307eb0644d85a1d1daaa61352697', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 22:34:53.341667 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dc7449-fdf0-4ef7-9055-197d5bcad0f6 service placement] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 22:34:53.342036 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dc7449-fdf0-4ef7-9055-197d5bcad0f6 service placement] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 22:34:53.342316 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dc7449-fdf0-4ef7-9055-197d5bcad0f6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 22:34:53.344300 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-b0dc7449-fdf0-4ef7-9055-197d5bcad0f6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 22:34:53.355661 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dc7449-fdf0-4ef7-9055-197d5bcad0f6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 22:34:53.357939 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dc7449-fdf0-4ef7-9055-197d5bcad0f6 service placement] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 22:34:53.368066 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1460/2919] 10.222.0.18 () {60 vars in 1365 bytes} [Sat Aug 6 22:34:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 22:35:03.363696 dv-cinder-849846-7-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=58 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}